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PREFACE 


This book contains introductory material on numerical methods of 
solving engineering problems by digital computers. The book is written 
for engineers and scientists who desire to achieve a working knowledge 
of digital computer calculations and programming methods for subse- 
quent application in their field of practice. For this reason, the book 
covers basic digital computer methods which can be applied to all 
fields of engineering, rather than specific applications. The level of 
the contents is such as to make the book particularly suited for self- 
study. Each chapter can be taken independently for the study of a 
specific subject or a specific application. 

The book can be used as a junior-senior level text for a one semes- 
ter course in digital methods given to engineering students from vari- 
ous branches of engineering or as a reference book to supplement the 
numerical computation portion of general applied mathematics courses. 

Vil 


Vill Preface 


It is assumed that the readers have had two years of engineering 
mathematics. No knowledge of computer programming is assumed on 
the part of the reader. Digital computer programs are given for a number 
of numerical examples to familiarize the reader with actual computer 
programming logic. These programs are written in FORTRAN (used by 
general purpose computers) and BASIC (used by time-sharing computers) 
programming languages. These programming languages are so close to 
actual algebraic statements that, with the explanatory paragraphs given 
with each program, the reader will have no difficulty following the 
logic of the computer programs. It should be noted that the digital 
computer programs are written primarily for simplicity of logic rather 
than efficiency of computations. 

The following is a general description of the contents of various 
chapters. 

Chapters 1 and 2 contain discussions and methods of numerical solution 
of determinants, matrices, and linear algebraic equations. Included in 
these chapters are methods of evaluating characteristic values and vectors 
using digital computers. A physical interpretation of characteristic values 
and vectors is given at the end of the second chapter. 

Chapter 3 includes a number of interpolation formulas and methods of 
curve fitting primarily used with statistical data. Polynomial root finding 
methods are also described in this chapter. 

Chapter 4 gives a discussion of time-frequency domain analysis using 
Fourier series and transforms and describes the use of digital computers 
in spectral analysis of time dependent functions. 

Chapters 5, 6, and 7 start by describing the basic relations between 
the differential and difference operators and methods of reducing a 
differential equation to a difference equation. Subsequent discussions 
deal with solution of various types of differential equations using differ- 
ence methods and digital computers. An appendix containing elementary 
solutions of typical differential equations is included for readers desiring 
a review of classical methods of solution of differential equations. 

Chapter 8 describes the numerical methods of solution of partial 
differential equations. Specifically, the numerical solutions of Laplace’s 
equation, the heat equation, and the wave equation are presented. 

Chapters 9 and 10 give a discussion of the elements of linear and 
dynamic programming. On the assumption that the majority of readers 
are not familiar with the subjects of these chapters, these chapters are 
written at a somewhat lower level than the rest of the text. 
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7 DETERMINANTS, 
MATRICES, AND LINEAR 
SIMULTANEOUS 
EQUATIONS 


The theory of determinants is intimately connected with that of 
matrices and with the solution of simultaneous linear algebraic equations. 
A great deal of the terminology and some of the operations involved 
in matrix algebra are based on a knowledge of elementary deter- 
minant theory. In this chapter we start with a brief discussion of the 
elements of the theory of determinants. This elementary discussion 
should provide an adequate review for the reader who is primarily 
interested in matrix algebra and matrix calculus from a standpoint 
of applications to engineering and physics. The rest of the chapter 
is devoted to matrices and numerical solutions of matrix equations 
and linear simultaneous equations. 
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DETERMINANTS AND SIMULTANEOUS 
EQUATIONS 


It is sometimes convenient to express the combination of terms (ad-bc) 
in the following form: 


G -< 
a = ad-bc (1.1) 


| 


D is called a determinant of the second order. The letters a, b, c, d, are 
called the elements of the determinant; (ad-bc) is called its expansion. 
The first row of D consists of the elements a and c. The second row of 
D consists of the elements b and d. The elements a and b comprise the 
first column and the elements c and d the second column. 

A determinant of the third order has three rows and three columns; 
a determinant of the order n has n rows and n columns. The transpose 
of a determinant D of any order will be denoted by D’; it is the deter- 
minant formed by interchanging the rows and columns of D. 

It is therefore evident that if 


(1.2) 


Determinants of the nth Order 


An nth order determinant A is a square array of n* quantities a; 
called the elements of the determinant written in the form 


67152: Mes oes Ay 
A = a51 99 G93 = Gon (1 3) 
4 a2 213 Gan 


Consider the following definitions of quantities with respect to the 
above determinant. 

Minors. If in the determinant A of Eq. (1.3) we delete the ith row 
and the jth column and form a determinant from all the remaining 
elements, we shall have a new determinant of (x — 1) rows and columns. 
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This new determinant is defined to be the minor of the element a; +. nae 
minor of the element a,, will be denoted by M, fe 

Cofactors. The cofactor of an element of a determinant a; ; is the 
minor M,. of that element with a sign attached to it. The sign is deter- 
mined by the numbers i and j which fix the position of a ij in the deter- 
minant A. The sign is chosen by the equation 


C;. = (1) 3M, (1.4) 


where Ci; is the cofactor of the element a, j and M ij is the minor of the 
element a5. 

In the general nth order case if C;; is the cofactor of any element a;; in 
the determinant A, the determinant can be expressed as the sum of the 
elements in any one row or any one column each multiplied by its co- 
factor. That is, 


acl. (any row 1) (1.5) 


J 


o 
I 


De ee (any column j) (1.6) 


z 


By repeating the expansion represented by Eq. (1.5) or (1.6), a deter- 
minant of large order n can be expressed in terms of determinants of 
steadily smaller order and hence can be evaluated. 

It is easy to demonstrate that, in general, 


. ae ee 
¥ 9, : ) GCs = ) ee 
: a APT rs 


It should be noted that the evaluation of determinants in terms of 
cofactors as given by Eqs. (1.5) and (1.6) involves excessive numbers 
of arithmetical operations. It can be shown that by using this method, 
the number of multiplications can be approximated by n! where n is the 
order of the determinant. For a determinant of order 10 this will result 
in 10! or 3.6 x 10° multiplications. In succeeding sections, methods 
using the fundamental properties of determinants will be given where 
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the number of multiplications is reduced to n?/3, or about 300 multipli- 
cations for evaluating a 10th order determinant. 


Fundamental Properties of Determinants 


From the basic formulas (1.5) and (1.6), the following properties of 
determinants can be deduced: 

1. If all the elements of a row or of a column are zero, the determi- 
nant is equal to zero. This may be seen by expanding in terms of the 
elements of that row or column in which the elements are equal to zero. 

2. If all elements but one in a row or column are zero, the determinant 
is equal to the product of that element and its cofactor. 

3. The value of a determinant is not altered when the rows are 
changed to columns and the columns to rows. In other words, the 
determinant and its transpose are equal. 

4. The interchange of any two columns or two rows of a determinant 
changes the sign of the determinant. 

5. If two rows or two columns of a determinant are identical, the 
determinant is equal to zero. 

6. If all the elements in any row or any column are multiplied by any 
factor, the determinant is multiplied by that factor. 

7. If each element in any column or any row of a determinant is 
expressed as the sum of two quantities, the determinant can be expressed 
as the sum of two determinants of the same order. 

8. It is possible, without changing the value of a determinant, to 
multiply the elements of any row (or any column) by the same constant 
and add the products to any other row (or column). 


Use of the Fundamental Rules in Evaluating 
a Determinant 


The fundamental rules enumerated above may be used in the numerical 
evaluation of determinants. For example, let it be required to evaluate 
the determinant 


(Cia ee 

iei=45 Bead 1.8 
ae (1.8) 

ee oe ee 

par tee 2 
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This determinant will be greatly simplified if we use the above properties 
to make all its elements except one in some row or column equal to 
zero. The presence of the factor —1 in the second column suggests that 
we add four times the first row to the second, then add two times the 
first row to the third row, and finally add the first row to the fourth row. 
By principle 8 these operations do not change the value of the determi- 
nant. Hence we have 


oe 
ie a 
Zo. t 
p= See cee | ee Coe 
SAA SG 
a 
ee ape ee 


We now subtract the elements of the last column from the first column 
of the third-order determinant of Eq. (1.9) and thus obtain 


vegies gaagee 


1.10 
Oe ee eee © See ) 


We now add two times the third row to the first row and two times the 
third row to the second row and thus obtain 


b--49 12 
40 19 (1.11) 
eS es Ae SS 
31 18 
= ee ee 


Now we may subtract the second row from the first row and obtain 


= sb? = BH == 1381 GAD 
ay 418 


The procedure described above is much shorter than a direct appli- 
cation of Eq. (1.5) or (1.6). 
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Chio Method for Evaluating Determinants 


_ The above procedure can be systematically applied to reduce the order 
of large determinants for numerical evaluation. In matrix algebra this 
procedure is referred to as the method of Chio. Consider the 4 x 4 
determmant 


> ye i ee 


ps (1.13) 


tS ey ee! oes 


The idea of this method is to reduce the order of the determinant from 
4 to 3 and from 3 to 2, and so on. This results in a 1 x 1 determinant 
which will give the actual value of the original determinant. To describe 
this, we proceed as follows. Make any element of the determinant, say 
element (1, 1), equal to unity by dividing the first row through by Q11- 
This results in 


O17 ig = ig 
or a 


D = a,,|%1 %%2 %3 44 (1.14) 


Denoting @)5 = @19/@11, 413 = 413/411, 214 = 4,4/a,, and multiplying 
the first row by a,, and subtracting from the second row, then multi- 
plying the first row by ag, and subtracting from the third row and, 
finally, multiplying the first row by a 41 and subtracting from the fourth 
row, we have 


: A492 a3 A14 

O G99 — 21419 Agg - 491443 Ag4 - 4p, O44 (1.15) 
0 439 — 431412 433 - A31%}3 434 - 431444 
0 


A49 -— 9419419 Ag - M4193 Aq - 41 O44 
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The value of this determinant now becomes 
Qi, 9 413 
” = 1 aan 7 y wy ” 
D’ = a,,(-D G31 a9 = 998 (1.16) 
Go, Bag O59 


where (-1)!*! is the sign of the element (1, 1) and double primes are 
used to denote elements 


oS ee ee: eee 5 
tT 22 i ae Bee A s2 pal pe Reset Ss Ee 
(1.17) 


51 = Ago - 43149, 939 = 433 - 431% 13>--- 


and so on. Thus Eq. (1.16) is a determinant of order 3 x 3. Repeating 
the above operation, this determinant will reduce to a 2 x 2 and subse- 


quently to a 1 x 1 with multipliers a,, aj, 47,---- Thus, the value of the 
original 4 x 4 determinant can be obtained. 
In cases where any of pivotal elements— a,,, a}, @{,, --- in the above 


derivation—become zero, the appropriate columns or rows should be 
interchanged to obtain a nonzero pivotal element. The process can then 
be continued. If an element which is already unity exists in a certain 
row, that element can be taken as the pivotal element, thereby reducing 
the number of divisions required to obtain a unity pivotal element. This, 
however, in a digital computer program will complicate the logic and will 
require additional logical statements. 

The total number of arithmetical operations* required in evaluating 
determinants by the method of Chio can be shown to be 


(n — 1)(2n2 — n + 6) 
6 


nGE = }) 
2 


multiplications 
(1.18) 


divisions 


*Multiplications and divisions rather than additions and subtractions are used as a 
basis for calculating the required digital computer computation time, since multipli- 
cations and divisions take far more (by a factor of 5 to 10) computation time in 
digital computers. 
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where n is the order of the determinant to be evaluated. Fora 10x 10 
determinant the above relations give, approximately, 275 multiplications 
and 45 divisions. Assuming that each operation will take 50 micro- 
seconds on a digital.computer, the total computation time will be 320 x 
50x 10° = 16 milliseconds. 


FORTRAN Program for Determinant Evaluation 


The logic of evaluating determinants by the method of Chi, as dis- 
cussed above, can be programmed for digital computer solution. Such a 
program, written in FORTRAN language, is given below. 


CSAH1 DETERMINANT EVALUATION PROGRAM 
10 DIMENSION S(100,100) 
20 DATA N/3/ 
30°. DATA (S(t. J) J=1,3),1-1,3N1.,2:3..3.4.8.2.1.4/ 


40 [=I 
50 K=2 
60 XM1=1. 


70 XM=S(L,L) 

80 DO 95 J=L,N 

90 S(L,J)=S(L,J)/XM 

95 CONTINUE 
100 DO 140 I=K,N 
110 X=S(1,L) 
120 DO 140 J=L,N 
130 S(1,J)=S(1,J)—-S(L,J) «X 
140 CONTINUE 
150 L=L+1 
160 K=K+1 
170 XM1=XM1*XM 
180 IF (L-N)70,190,190 
190 XM1=XM1#S(N,N) 
200 WRITE (2,210) XM1 
210 FORMAT (22H VALUE OF DETERMINANT= 1PE10.3) 
220 STOP 
230 END 


The computer solution obtained is 


VALUE OF DETERMINANT = 1.000E00 


Determinants, Matrices, and Linear Simultaneous Equations 9 


In order to completely understand the above program the reader should 
consult the FORTRAN reference manual listed as [1]. The logic of the 
program, however, can be explained by considering the evaluation of the 
following 3 x 3 determinant used in the program. It is required to 
evaluate the determinant 


\ eae ee 
5 ae (1.19) 
pa, ae 


Statement 10 of the program will reserve computer memory space for a 
100 x 100 determinant. Statements 20 and 30 will read into the com- 
puter memory the order of the determinant N = 3 and the elements of 
the determinant S(,J), starting from the first row and proceeding to 
the second and the third rows. Statements 40 through 60 will set indices 
and the value of XM 1 = 1.0 to be used later in the program. Statement 
70 will set XM equal to the value of element (1, 1) or S(1, 1) of the 
determinant or in this case S(1, 1) = XM = 1.0. The loop consisting of 
statements 80 through 95 will divide the elements of the first row by 
XM. In this case, since XM is equal to 1, determinant of Eq. (1.19) will 
be unchanged. The double loop consisting of statements 100 through 
140 and 120 through 140 will multiply the first row by element S(2, 1) 
and subtract the resulting first row from the second row and will repeat 
this process with the third row. At the end of this operation, the 
resulting determinant will be 


SS ee 
ea Ss (1.20) 
0 -3 -95 


Statements 150 and 160 set in indices L= L+ 1 and kK =K +1, resulting 
in L = 2 and K = 3. The value of XM1 is equal to previous XM 1 times 
XM, or at this point XM1= 1.0. Thus, at statement 170 the determinant 
is reduced to a second-order determinant 


-2 -3 
(1) X 


CLE) 
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Statement 180 checks to see if the value of L is smaller than the order of 
the determinant, in which case the above operation starting with state- 
ment 70 is repeated. The second application of logic described by 
statements 70 through 170 will result in 


ee BS 


(1) (- 2) (1.22) 
0 -.5 


This time, since N will be equal to L at statement 180, the logic will go 
to statement 190. This will set 


XM1 = (1)(—2) (-.5) = 1 (1.23) 


which is the value of the determinant as printed in the computer solution. 
Note that in the above program the 3 x 3 determinant was kept in 
computer storage while its elements were changed and operated upon 
without manipulating the location of unused elements. 


The Solution of Linear Equations; 
Cramer’s Rule 


A general system of n simultaneous linear equations can be written in 
the form 


Q414X1 + AyoX_ + --- + A),xX, = Ry 
Qo1X1 + AgoXy +... + Gp, X, = Ry (1.24) 
@4%1 + 4.9%, +--+. + a,x, = Rk, 


The solution of the system (1.24) can be shown to be given by 


r 


a Se, es, 
D (1.25) 
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where D is the determinant of the coefficients of Eqs. (1.24) 


Oyj Uo eS Qin 
po a ae ee (1.26) 
a4 a2 a3 Ann 


The determinant D, is obtained by replacing the rth column of D by 
(Ry, Ro, Rg, ..-, k,). The result (1.25) is called ““Cramer’s Rule” in the 
mathematical literature. It presents an elegant but laborious method 
for solving sets of simultaneous equations. 

From Eq. (1.18) for large values of n, high-order determinants, the 
number of multiplications can be approximated by n?/3 and divisions 
by n2/2. Assuming the number of divisions to be negligible as compared 
to the number of multiplications, the number of arithmetical operations 
in evaluating nth order determinants by the method of Chio can be set 
at n?/3. Using Cramer’s rule in solving n simultaneous equations in n 
unknowns, n + 1 determinants should be evaluated. Using Chid’s 
method for the evaluation of these determinants will result in the total 
number of arithmetical operations (n?/3) (n + 1) = n4/3. In succeeding 
sections it will be shown that Gauss’ method of elimination requires 
only n°/3 multiplications in solving n simultaneous equations in n un- 
knowns. For this reason Cramer’s rule should be taken more for its 
historical and academic value rather than for its value in practical 
application. 

The Regular Case. If the determinant of the coefficients D of the 
equations does not vanish so that D # 0, the set of equations is said to 
be regular. In this case they have the unique solution (1.25). 

The Singular Case. If D = O, the set of equations is said to be singular. 
A singular set of equations may or may not be consistent. The question 
of consistency will not be considered here. However, it may be pointed 
out that in simple cases the question of consistency can be decided by 
trial. | 


Homogeneous Equations in 7 Unknowns 


If all the numbers k,, aS Se in Eqs. (1.24) are zero, the set of 
equations is said to be a set of homogeneous equations. It can be proven 
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that the necessary and sufficient condition that a set of homogeneous 
equations should have nonzero solutions (x, x9, X35, .--, x, not all zero) 
is that the determinant of the coefficients D should vanish, or D = 0. 

An important relation between the ratios of the unknowns in the 
case of nontrivial solutions can be established in the following heuristic 
manner. Let it be assumed, for the moment, that the determinant D is 
not zero and that only one of the k’s in Eqs. (1.24), say k,, 18 different 
from zero. In this case by Cramer’s rule we would have 


see : (1.27) 


The result (1.28) is independent of the values of both k. and the 
determinant D. It may be inferred, therefore, that Eq. (1.28) holds also 
when both k, and D are zero. The correctness of this conclusion can be 
demonstrated in a rigorous fashion. It is therefore interesting to note 
that when the set of homogeneous equations has nontrivial solutions, 
the ratios of the unknowns are fixed by Eq. (1.28) in terms of the 
ratios of cofactors of the determinant D. 


MATRIX ALGEBRA, LINEAR 
SIMULTANEOUS EQUATIONS 


Matrices provide a compact and flexible notation that is particularly 
useful in studying linear transformations, and present an organized 
method for the solution of systems of linear algebraic and linear differ- 
ential equations. Since many engineering problems are formulated in 
terms of linear algebraic or linear differential equations, it is natural that 
their solution will be expedited by the use of matrix algebra and matrix 
calculus. Matrix algebra has been found so useful in pure and applied 
mathematics that it has been termed “the arithmetic of higher mathe- 
matics.” 
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Definition and Notation 


A table of n by m quantities, called elements, arranged in a rectangular 
array of n rows and m columns is called a matrix with n rows and m 
columns. If a,, is the element of the ith row and the jth column, then 
the matrix A can be written in the following manner: 


ay, Oyo Q1m 
a a a ere 6: 

Pape ae 22 23 2m (1.29) 
an) a2 2,3 Qnm 


The matrix A of Eq. (1.29) is said to be a matrix of order (n,m). If the 
number n of rows is equal to the number m of columns, then the matrix 
A is called a square matrix. 

Besides square matrices, two other general types of matrices frequently 
occur. One is the row matrix R: 


Sy tt r | (1.30) 


The other is the column matrix C 


eee (1.31) 


Elementary Operations on Matrices 


Transposition. It is possible to interchange the rows and the columns 
of the matrix A of Eq. (1.29) to obtain a new matrix A’ of the form 


20 ee 

<= es eee 2,2 
es Seer aS. (1.32) 

Qim om a3m anm 
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The matrix A’ is called the transpose of A. The transpose of the row 
matrix Eq. (1.30), R’, is a column, and the transpose of the column 
Eq. (1.31), C’, isa row matrix. 

Addition of Matrices. Let A and B be two matrices of the same order, 
that is, they have the same number of rows and columns. Let 


A=[a,J, B= (b,J (1.33) 


Then by the sum A + B of the matrices A and B we shall mean the 
uniquely obtainable matrix C whose elements c jj are obtained by the 
equation 


f= 2S 


ey 
ia Se 


C22 AE (1.34) 


In other words, to add two matrices of the same order, we calculate 
the matrix whose elements are precisely the numerical sum of the 
corresponding elements of the two given matrices. The addition of two 
matrices of different orders is not defined. 

Equality of Matrices. To complete the preliminary definitions, we 
must make clear what is meant when two matrices are said to be equal. 
Two matrices A and B are defined to be equal provided that they have 
the same order and that their corresponding elements are equal. That is, 


=f. fe = §. (1.35) 


From the above definitions of the sum of two matrices and the equality 
of two matrices it follows that addition of matrices is associative, that is, 


BoA + 0) 44 + Be CS A ee (1.36) 


Before proceeding with the definition of multiplication of matrices, a 
word or two must be said about two very important special square 
matrices. One is the zero matrix—a matrix all of whose elements are 
zero, for example, 
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0 ee () 
0-10 0 0 Of, a zero matrix of order (3, 4) (1.37) 
0 0) 0) 0 


Another matrix of great importance is the unit matrix. This is a square 
matrix of any order whose principal diagonal elements are unity and has 
zeros for all its other elements; it is denoted usually by the letter I. 

For example, the unit matrix of the fourth order is 


1 

a a (1.38) 
6 22s 
eee 


Multiplication of Matrices 


Multiplication of a Matrix by a Scalar. By definition, the multiplica- 
tion of a matrix A by an ordinary number or scalar k is effected by 
multiplying each element of the matrix A by the number k, thus obtain- 
ing a new matrix whose elements are ka; .. That is, by definition, 


B = kA where 0O.. = ka.. (1.39) 


For example, 


2 4 6 12 
ae eee 


The Multiplication of a Matrix by Another Matrix. The definition 
of multiplication of one matrix by another has been chosen in order to 
facilitate operations involving linear transformations by the use of matrix 
algebra. 

The rule of multiplication is such that two matrices A and B can be 
multiplied together in the order AB only when the number of columns 
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of A is equal to the number of rows of B. That is, if A is a matrix of 
order (m,n) and B is a matrix of order (p,q), the product AB is not 
defined unless n = p. If n = p, the matrices A and B are said to be con- 
formable. In matrix algebra only conformable matrices can be multiplied 
together. 

The product of a matrix A of order (m, p) by a matrix B of order (p, n) 
is defined to be the matrix C given by 


C = AB = [e,,] (1.41) 


where the elements Ci; of the matrix C are obtained by the equation 


k=p 
2 
29 a, (1.42) 


It can be seen that the product matrix C is a matrix of order (m,n). The 
fact that a matrix of order (m, p), when multiplied by a matrix of order 
(p,m), results in a matrix of order (m,n) may be expressed symbolically 
in the following manner: 


(m, p)(p,n) = (m,n) (1.43) 


This symbolic equation is useful in determining the order of the 
resulting matrix when several matrices are multiplied together. As an 
example of the definition of the rule of multiplication Eq. (1.42), let 


Orr Oy. Sy bi, = dy5 
A= 145; 99 3 B= |b 9, 499 (1.44) 
G3, 39 433 b3, b39 


In this case we have matrices of orders (3, 3) and (3, 2). They are 
therefore conformable and the product matrix C is of order (3, 2). In 
this case the typical element c ij of the product C = AB is given by Eq. 
(1.42) in the form 


Ci; = 515); + Ain bg, + Aj3 63, (1.45) 
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The elements Eq. (1.45) are now arranged in the rectangular array, 


(444 yy + Gy 945, + 5534) (yy Hyg + 29599 + 413 O39) 

C = | (ag, by1 + Ago boy + Ag3b31) — (Ag1 Bg + Ag2 bog + G23 O39) 
(231 51 + Ag9b91 + 433534) (gy yg + G39 bog + 433 O39) 
(1.46) | 


where C is the product matrix AB. It can be seen that the result Eq. (1.46) 
can be obtained by taking the sum of continued products of the rows of 
A times the corresponding columns of B. 

Properties of Matrix Multiplication. In general, matrix multiplication 
is not commutative, that is, 


AB # BA (1.47) 


This can be seen directly from the definition Eq. (1.42). The matrices, 
Eq. (1.44), are not conformable in the order BA and cannot be multi- 
plied in this order. 

In matrix algebra it is necessary to distinguish between premultiplica- 
tion, as when A is premultiplied by B to give the product BA, and post- 
multiplication, as when A is postmultiplied by B to give the product 
AB. If we have the equality 


' AB = BA (1.48) 


then the matrices A and B are said to commute or to be permutable. The 
unit matrix I, it can be noted, commutes with any square matrix of the 
same order. This is, 


IA = Al (1.49) 


provided that I and A are of the same order. 

Continued Products of Matrices. Except for the fact that matrices do 
not commute in multiplication, all the ordinary laws of scalar algebra 
apply to the multiplication of matrices. Of particular importance is the 
associative law of continued products: 


ABC = -A(BC)S (ADC (1.50) 
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The fact that the associative law is satisfied permits one to dispense with 
parentheses and to write ABC without ambiguity. It must be noted, 
however, that the product of a chain of matrices will have meaning only 
if the adjacent matrices of the chain are conformable. 

The Reversal Law of Transposed Products. It can be shown as a con- 
sequence of the definition of the rule of matrix multiplication Eq. (1.42) 
that the transpose of a product of matrices is equal to the product of 
their transposes in reverse order. That is, if we have 


M-= AB OM = BA’. (reversed order) (1.51) 


where primes indicate transpose matrices. Similarly, if 
M = ABC then M = CBA 52) 
This result is called the reversal law of transposed products. 


Partitioned Matrices and Partitioned 
Multiplication 


It is sometimes convenient for purposes of computation to extend the 
use of the fundamental laws of combinations of matrices to the case in 
which a matrix is considered to be constructed from elements that are 
themselves matrices. For example, consider the matrix A 


Oty yo 
| ro 
A-24 “92 fee px = oe (1.53) 
Stas. jaoees Bs Ta 
Qs, a39 =| agg 


The matrix A above has been partitioned by the introduction of a vertical 
and a horizontal line into the four submatrices P,Q, R, and S given by 


a a a R = [@9, Gao] 
yp fo 12 i. 13 
i ae 293 S = dg 
In this case, the matrix A has been partitioned in such a manner that 
the diagonal submatrices P and S are square, and the partitioning is 


Determinants, Matrices, and Linear Simultaneous Equations 19 


diagonally symmetrical. Now let B be another square matrix of the third 
order partitioned similarly to A in the form 


fig ae : O13 
i, 
eg oo eee eee : : (1.54) 
ee ee 


Now it can easily be seen that the sum C = A + B can be expressed in 
terms of submatrices in the form 


ie Ps ) 
Wee ee: ei ae (1.55) 


fo vie ae oeacee ara 


This indicates that when two matrices of the same order are partitioned 
similarly, their sum can be obtained by adding their various submatrices 
as if they were elements. 

Multiplication in Terms of Submatrices. As a consequence of the fun- 
damental rule for the multiplication of matrices, a rectangular matrix B 
can be premultiplied by another rectangular matrix A, provided that the 
two matrices are conformable, that is, when the number of rows of B 
equals the number of columns of A. Now if A and B are both parti- 
tioned into submatrices such that the grouping of columns in A agrees 
with the grouping of rows in B, it can be shown that the product AB 
can be obtained by treating the submatrices as ordinary elements and 
proceeding according to the general rule of multiplication. 

In the case of the matrices A and B discussed above, the partitioning 
is such that the product D = AB can be carried out by treating the 
submatrices of Eqs. (1.53) and (1.54) as if they were ordinary elements, 
thus obtaining . 


D = AB (1.56) 


(PP, + QR,) (PQ, + QS,) 
MRP, 4 SRi) «ARQ, ..S8)) 
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Matrix Division; the Inverse Matrix 


Having defined the addition and the multiplication of matrices, we 
now consider an operation that is analogous to the operation of division 
in ordinary scalar algebra. In ordinary algebra, if the product of two 
quantities b and a equals unity, so that 


ba = 1 CL37) 


we say that b is the reciprocal of a and we write 
pa (1.58) 
a 


In matrix algebra the unit matrix I of Eq. (1.38) has properties similar 
to that of the number one in ordinary algebra since Al = IA = Aif Aisa 
square matrix of the same order as /. If we now have two square matrices 
A and B of the same order and if they satisfy the equation 


BA = I! (1.59) 


we see that Eq. (1.59) is the matrix analog of Eq. (1.57); we then say that 
B is the inverse matrix of A and we write 


BoA gpd A A = I (1.60) 


Equation (1.60) is the analog of the scalar Eq. (1.58). 

It can be shown that the necessary and sufficient condition for the 
inverse of matrix A is that the matrix A be nonsingular, that is, |A| # 0. 
In order to find the inverse of matrix A, let us define the adjoint of A 
as the transpose of the matrix formed by replacing the elements of A by 
their cofactors C, - For a (3 x 3) matrix, this is written as 


C 
adjoint (A) = adj A =|C,, Cyn Cap (1.61) 
C 
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On the strength of Eq. (1.7) we can write 
(A)(adj A) = D{I] (1.62) 


where D(= |A|) is the determinant of matrix A and [J] is the unit matrix. 
From the above equations the inverse matrix can be written 


ae (1.63) 
ae 
Or 
: A 
At. me (1.64) 


In practice the inverse of matrix A is formed by obtaining the transpose 
of matrix (A’) and subsequently obtaining the matrix of the cofactors 
of A’. Division of the matrix of the cofactors by the value of the 
determinant of A(= |A\|) will give the inverse matrix A‘. 

Although the above method of matrix inversion represents the classical 
approach, computationally it involves an excessive number of arithmetical 
operations. It has been shown that evaluation of large-order determi- 
nants by the method of Chio involves, approximately, n>/3 multiplica- 
tions, where n is the order of the determinant. Using this as a basis, 
evaluation of the inverse of ann by n matrix by the method of cofactors, 
described above, will involve evaluation of n* cofactors of order (n — 1) 
by (n — 1) and, in addition, the determinant of the matrix, resulting in 
(n2)(n — 1)3/3 + n3/3 multiplications. For large n the above can be 
approximated by n°/3 multiplications. In the following sections it will 
be shown that matrix inversion, for example, by the aigmented matrix 
method, can be accomplished by as few as n? multiplications, a reduction 
by a factor n2/3 from the number of multiplications required by the 
cofactor method of matrix inversion using the method of Chio in 
evaluation of the cofactors. 


Gauss’ Method of Elimination in Solving 
Simultaneous Equations 


This method is probably the simplest approach to the solution of a 
series of simultaneous equations. It consists of eliminating one unknown 
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at a time from the set of equations and proceeding with the remaining 
equations. This, at the end, will result in only one unknown and one 
equation. Working backward from this equation, one can solve for one 
unknown at a time until the solution is complete. Consider the set of 
equations 


€y4X1 + AjQX%o + Ayg%3 + +--+ + A,X, = by 
Ao1X1 + AggoXg + AggXg + +++ + Ap X, = bo 
ee mE Pe ee es ks ee (1.65) 
@izM + AigXq + AigxXg + + G5, = 2; 
an1™ +o 9,3 4 a ee b, 


Solving the first equation for x,, we get 


Substituting this value of x, in the second through nth equation, we 
obtain a set of n — | equations in n— | unknowns x, through x,. This 
set can be written 


Ao9Xq + AggXg + A5,%, = 9 

, 2 } 1.66 
dj;9X9 a Wg hq o+ QinXn = b; ( ) 
A gXo + A" 3X + Bin®n = b 


where primes denote the new coefficients. Now, solving the first 
equation for x, and substituting in the second through nth equations, 
we obtain a set of n — 2 equations in n — 2 unknowns x5 through x,. 
Continuing this operation will result in two equations in two unknowns 


C- % cae od C= 
n-1,n-1"n-1 2 eee ee aay n=] 
(1.67) 
On n= = A nn d,, 


Determinants, Matrices, and Linear Simultaneous Equations 23 
and finally in one equation and one unknown 
pHa age (1.68) 


where c’s and d’s represent constants. Thus, Eq. (1.68) will give the 
value of x,. Having x,, Eq. (1.67) will give the value of x, _,. Continu- 
ing this process will result in values of x, 5, %,3.--+-+» %g and xj. 

It can be shown that the number of multiplications and divisions 
required in the above method is given by 


not a 1)i2e a) 
6 


multiplications 
n divisions 


where n is the number of simultaneous equations to be solved. For large 
n, the above values (neglecting divisions) can be approximated by ners. 
a factor of n advantage over the solution by Cramer’s method using 
Chid’s method for the evaluation of the determinants. 


Triangulation Method in the Solution of 
Simultaneous Equations 


The triangulation method is often used in obtaining the solution of a 
set of simultaneous linear equations. This method primarily consists of 
systematically reducing the number of unknowns from each equation, 
and ending up with a single equation and a single unknown. One can 
solve this equation for the unknown and work backward to solve for all 
the rest of the unknowns one at a time. To describe this in equation 
form, consider the set of linear equations 


@44% + @poX%e + «> + 04,%,, = by 
(1.69) 
Byte F Gigkg + + 6X, = b. 


Ck aw Ratt aa, ag eg Fe b 
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Divide through the first equation by a,,. Multiply this equation by Q51 
and subtract from the second equation. Multiply the first equation by 
a3, and subtract from the third equation. Follow this procedure up to 
the nth equation. This will result: 


oy A19Xo bears Q11Xp = b4 
@ogXo + +++ + Ag, X, = b5 
(1.70) 
DigXq + r++ + sae = b- 
Soa + Gs = b, 


where primes denote new coefficients. 

Now divide the second equation by a5, and subsequently multiply the 
resulting equation by a35, a45,-.. and subtract from the third, fourth, 
..., equations, respectively. This operation will eliminate unknowns Xo'S 
from all of the equations except the first and the second equations. 
Continuing this procedure will triangulate the set of equations resulting 
in 


X41 + C19%X9 + C13%3 3 eee es C1, = d, 
X9 + Co3 XQ ae i ne Cs, Xn = dy 
Se (1.71) 
+= 4 © OC peqee n-1l 
x = d 
n n 


where c’s and a’s are constants resulting from arithmetical operations. The 
last equation gives the value of x,. This x, can be substituted in the 
equation before last resulting in the value of x,- 1. Continuing this 
process, all of the unknowns can be obtained. 

The triangulation method can be recognized as a systematic procedure 
to effect Gauss’ elimination method of solving simultaneous equations. 
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Since the logical operations of the triangulation method are straight- 
forward it can easily be programmed for digital computer solution. 
The number of multiplications and divisions required is approximately 
the same as that of Gauss’ elimination method. 

Example. Consider the solution of the set of equations in unknowns 
x, y, and z by the above method. 


<4 By + 2G 
ee Fe 
ee 9 oe Se ( ) 


g = 9 + 72 = 6 


Multiplying the first equation by —2 and adding to the second and 
multiplying the first equation by —1 and adding to the third will result in 


Ea Sy 4 2 Se 4 
Se ee 


—~3y+ z= -7 
Dividing the second equation by —3 we get 


<4 2 g- = 4 
ype = 41 


—~3y+2= -7 


Multiplying the second equation by 3 and adding to the third will result 
in 


tay s+ ee A 
ee ee 
4z = -4 
The last equation gives z =—1l. Having this, the second equation gives 


y = 2 and finally the first gives x = 1. 
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Solution of Simultaneous Linear Equations 
by Matrix Inversion 


Consider the set of n simultaneous equations in n unknowns: 


44X14 + AyoXg + AygXz + +++ + Ay,%X, = Oy 
Qg1X1 + AggXq + AggxX3 + +++ + Ag,xX, = dg 
(1.73) 
G;1%1 + GigXo + GjgXg t+ +++ + A,X, = 0; 
@_ 4%) + G_oX%o + O,g%q + *+ + 05%, = b., 


where a’s are the constant coefficient, x’s are the unknowns, and b’s 
are constants. The above equation can be written in matrix form as 
follows: 


[A] (x) = (B) (1.74) 


where [A] is a square n x n matrix, and (x) and (B) are column matrices. 
Matrices [A], (B), and (x) can be written 


i ee 2s = 


Al = 21 Ago --- Ag, (B) - °2 ae *2 (1.75) 


any Gg +++ 0 b % 


Premultiplying both sides of Eq. (1.74) by [A]~! results in 
ay = [Al A) (1.76) 


Thus the solution can be obtained by evaluating the inverse matrix 
[A]! and multiplying it by column matrix B. 

In previous pages, numerical methods for the solution of simultaneous 
equations were described, where the number of multiplications required 
for solving a set of n equations in n unknowns was, approximately, n?/3., 
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In the following pages matrix inversion methods, to be used in conjunc- 
tion with the above method, will be described where n> operations are 
required to invert an n x n matrix. Thus, the method of Eq. (1.76) will 
involve three times more multiplications in inverting the matrix A than 
previous methods require in solving for a set of n unknowns. For this 
reason the method of Eq. (1.76) should be applied only when a set of 
x’s for a set of B’s are to be calculated. Under these conditions the 
inverse matrix A can be reapplied to calculate new x’s for new B’s. 


Augmented Matrix Method of Matrix 
Inversion 


The augmented matrix method of matrix inversion is based, primarily, 
on the Gauss elimination method of solving simultaneous equations. 
For this reason its derivation will be based upon a set of n equations 
and n unknowns. It will be shown that the procedure of inverting 
matrices by this method can be mechanized without relating it to a set 
of simultaneous equations. 

Consider the set of equations 


a1, = a9 X95 5 eee QinX*n = by 
4S oa te FO = 
ott Spe aa 2nn 2 
(1.77) 
Gn1%1 + Gng%g t+ *** + Onin n > b, 


It is desired to find the inverse of the matrix of the coefficients [A]~!. 
This set of equations can be written in augmented matrix form as follows: 


ee Ay a 
ieee ee ae ear 0 || *2 
Gg ie ee a, LD 0 
x | 
eS 
SS on oe ee ae 
es 
6. 6 CU ast 1 : 
=O 
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where the large matrix is the augmented matrix of A. The idea of this 
method is to make all of the elements along the diagonal of the part A of 
the augmented matrix (a,,, ayo, .-., a,,,) equal to unity and the rest of 
the elements of part A equal to zero. In doing this the A, part of the 
matrix will fill up, resulting in 


a Sag sat os was elsh css reas 
8 

. 1 : 

1 : 
"a |. 9 G9) 

Numbers =p, 

—by 

1 = 

n 


Multiplying these matrices and transforming the (x) and (B) matrices to 
proper sides of the equal sign, we have 


“4 Ay 

AS BS Numbers (1.80) 
b 

x n 


Comparing this to Eq. (1.76) we see that [A]"' = A,. Substituting the 
values of b’s from the given set of linear equations and multiplying the 
two matrices on the right of Eq. (1.80) results in values of the unknown 
x’s. Matrix A, in Eq. (1.79) is obtained from the augmented matrix of 
Eq. (1.78) by the following procedure. 

In order to follow the method described here, relations (1.78) should 
be considered as a set of equations rather than as two matrices multiplied 
together. Divide the first row, of the first equation, by a,,. Multiply 
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the resulting values of the first row by a,, and subtract from the second 
row; multiply the first row by ag, and subtract from the third row. 
Continuing this will result in a matrix of the form 


#3 

_ ~~ o === 0s o Xo 

Goo = = a 0 : 
ey 

= = = Se —b, 

— bo 

— — a” a @ 1 

nn : 

—b 


where dashes and primes represent constants. Now, divide the second 
row by aj. Multiplying the resulting row by aj,(i =3, 4,..., m) and 
subtracting from the third row and higher will give zeros in the second 
column. Continuing this operation on the third, fourth, and up to the 
nth rows will result in the matrix 


-~ 


1 = 
1 - "2 
1 
eg oh 
== 
| ah 
; oe 


Working the above procedure backward from the nth row to the first 
row, i.e., multiplying the nth row by the element (n — 1, n) and 
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subtracting from the (n — 2) row, and so on, will produce zeros in the 
nth column. Repeating this process for the (n — 1)th row through the 
first row will result in the matrix 


a | 


‘ 
1 2 
Numbers 1 =a (1.83) 
Sy 
—b5 
: _b 
nm 


Comparing this and Eq. (1.79), we note that the square matrix “‘numbers”’ 
is the inverse of the matrix of the coefficients (or = [A]~‘). 

As mentioned before, the number of multiplications in inverting 
matrices by the augmented matrix method is estimated at n? where n is 
the order of the matrix to be inverted. 

In order to crystallize the above procedures, let us consider the follow- 
ing numerical example. 

Example. Find the inverse of the matrix 


ra 


a 


eae eee | 
ea (1.84) 
a 


by the augmented matrix method. In numerical evaluation the matrices 
(B) and (x) can be dropped and the mechanics of arithmetical operations 
described above can be carried out. The problem starts by writing 


oe ES (1.85) 
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Dividing the first row by 2, we get 


1 3 : 0 1 0 (1.86) 


Multiplying the first row by —1, adding to the second row, multiplying 
the first row by —1, and adding to the third row will result in 


1 Oe Dae ae DE 0 
0 2 a 8 0 (1.87) 
0 0.5 3h. =04. 8 1 


Dividing the second row by 2.5, multiplying the resulting row by —0.5, 
and adding to the third row, we get 


1 a ee 2 ee 0 
1 1 eee ey eee eee (1.88) 


So 


0 0) 3.4 -0.4 -0.2 1 


Dividing the last row by 3.4, multiplying the resulting row by —0.2, 
adding to the second row, multiplying the third row by —0.5, adding to 
the first row, we get 


1 05. 0 0.5588 0.0294 -—0.1470 
0 1 0  -0.1764 0.4118 —0.0588 (1.89) — 
0 0 1  —0.1176 —0.0588 0.2941 


Multiplying the second row by —0.5 and adding to the first results 
in 
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+ © 6 06171 6.1705 -C 76 
G6 1 0 -O0. 1764 0.4118 —O.588 | (1.90) 
626.4 Fw - 3355 ae 


From this the inverse matrix of the original matrix will be 


R04 obs 0.6471 -—0.1765 —0.1176 
Sie ee | =T 01764 ~~ 0 4Tis- Oss (1.91) 
ee ee 0.1176 —0.0588 0.2941 


FORTRAN Program for Augmented Matrix 
Inversion Method 


The following FORTRAN program is written for matrix inversion by 
the augmented matrix method discussed above. The data used in the 
program are the same as the example problem above. The computer 
program output is given at the end of the program listing. 

In order to understand the statements of the program, the reader 
is referred to [1]. The following paragraphs give a general explanation 
of the program. 

Statement 10 reserves computer memory location for a matrix of 
100 by 100. Statements 20 and 30 read the order of the matrix to be 
inverted, N = 3, and the matrix coefficients S(I, J), starting with the first 
row and proceeding to the second and third rows, as given in Fq. (1.84). 
Statement 40 and the following two statements set values to be used 
later. Statements 50 through 80 form the augmented matrix as given by 
Eq. (1.85). Statements 90 through 220 transform the augmented matrix 
into the form given by Eq. (1.88) with the last row divided by 3.4. 
Statements 230 through 290 transform the matrix into the form given by 
Eq. (1.90). Statements 300 through 330 print out the inverse matrix 
as given at the end of the program and also by Eq. (1.91). 


C MATRIX INVERSION BY AUGMENTED MATRIX METHOD 
10 DIMENSION S(100,100) 
20 DATAN/3/ 
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30. DATA ((S(I, J), J=1,3),1=1,3)/ 2.,1.,1.,1.,3.,1.,1.,1.,4./ 
40.) 


NX=N+1 
NY=2*N 

50 DO 80 J=NX,NY 

60 = S(I,J)=1.0 

70 

80 CONTINUE 

g0 be7 

Tuo KZ 


110 XM=S(L 

120 DO 140 J=L,NY 

130 = S(L,J)=S(L,J)/XM 

140 CONTINUE 

150 §=~DO 190 I=K,N 

160 X=Si(I,L) 

170. DO 190J=L,NY 

180 = S(I,J)=S(1,J)—S(L,J) *X 
190 CONTINUE 


200 —-L=1+1 

210 K=K+1 

220 ~=IF(L—N) 110,110,230 
230 L=N 

235 LZ=L-1 

240 DO 290 K=1,LZ 

250 =I=L—-K 

260 Y=S(1I,L) 


270 DO 290 J=L,NY 

280 = S(I,J)=S(1,J)—S(LJ)*¥ 

290 CONTINUE 

ooo b=] 

310 IF(L—1)320,320,235 

220 WRITE(2,330) ((S(1,J),J=NX,NY),1=1,N) 

330 FORMAT (15H INVERSE MATRIX / (3X1P3E10.3)) 
340 STOP 

350 END 


Computer solution obtained: 


INVERSE MATRIX 
6.471E—01—1.765E—01—1.176E—O01 
—1.765E—01 4.118E—01—5.882E—02 
—1.176E—01—5.882E—02 2.941E—01 
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Inversion of a Matrix by Partitioning 


The inversion of large matrices can be accomplished by partitioning 
the matrix into submatrices. This method reduces the order of the 
matrices to be inverted, thereby reducing the computer memory storage 
requirements. Consider the nth order matrix 


A|lB 
M = (+) (1.92) 
Gap 


partitioned into four submatrices A, B,C, and D where A and D are square 
matrices of order p and gq, respectively; p + q = n. The inverse matrix is 
denoted by 


= W | X 
M -(-+--) € E33) 


where W and Z are square matrices of orders p and qg. Using the 
property of inverse matrices 


AL ae Ee eee 
mM-1 (-4-)- +=). ( (1.94) 
el ph\y | 2 oo 


where I is the unit matrix and multiplying Eq. (1.94) we get 


AW4+-BY — ] 
AX + BZ = O 
(1.95) 
St Se. 2) eee &. 
b+ DZ 2 


Multiplying the third equation by BD~! and subtracting from the first, 
we obtain 


(A - BD-'Q@w =I (1.96) 
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or 
WA Bi (1.97) 
From the third equation we also find that 
y= Ai (1.98) 


From the second and fourth equations we obtain 


i 2A aS 
(1.99) 


Pie eet A 


From the above equations, it is seen that the inversion of a matrix of 
order n reduces to the inversion of two matrices of order p and two of 
order g and several matrix multiplications. 

The equations given above can be altered so that for computation of 
matrices W, X, Y, Z, only two matrices need be inverted. In terms of the 
inverse of the square submatrix D, we get 


1A BESS 

X-2 wep" 

: ee (1.100) 
Zt = Bee 


The above equations can also be set in terms of A’! matrix, as follows: 


ee ee 
2 goa! 

5A ee 
oA key 


(1.101) 


See ee oN 
| 


Note that Eqs. (1.100) and (1.101) involve the inverse of matrices D and 
A since they were the only square matrices assumed in the partitioning 
process. In the partitioning method of matrix inversion, since a great 
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number of calculations are involved, it is expedient to check the inverse 
matrix by multiplying it by the original matrix and obtaining the unit 
matrix. 

Example. Find the inverse of the third-order matrix 


5 48-4 26 
| 
not ot 42 (1.102) 
ee eee 
LO 32 2 o7 


by partitioning, as indicated. 
Partitioned matrices A, B, C, and D of Eq. (1.92) can be written 


oS ee 9.6 ; (1.103) 
a Bex ( C = (1.0,3.2) D = 6.7) 
= 2 ee 4.5 


From this, the inverse of matrix D of Eq. (1.100) will be 

pol 2 ) . 6 igs (1.104) 
6.7 

The matrix W of Eq. (1.100) will become 

2 75 3a 1.43280 4.58496|\! 
-\i-91 131] (o67ige2 2.1498 
( 1.06716 penne | 
2.77164  —0.849254 


(1.105) 


( 0.275531 = - 9 
-~0.89923 —0.34623 


Having the above matrix, the second of Eqs. (1.100) can be used to 
calculate X. This results in 
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( 0.275531 Pent 
— 0.89923 —0.34623 4.5 


as 5 

~ |. 1.52099 

Using the previously calculated values, the third and fourth relations of 
Eq. (1.100) will result in values of Y and Z as follows 


(0.14925) 


Y = (0.388359 0.20338) 
Z = —0.543799 


Setting the values W, X, Y, and Z in Eq. (1.93) of the inverse matrix, we 
get 


0.275531 —0.254709 —0.223718 
M-! — {0.89923 ~0.34623 1.52099 
0.388359 0.20338 0.543799 


(1.106) 


which is the required answer to the problem. 


Gauss-Seidel Method of Solving 
Simultaneous Equations 


A system of linear equations is called diagonal when the coefficient of 
the diagonal element dominates, that is, the absolute value of this coef- 
ficient is larger than the sum of the absolute values of the rest of the 
coefficients on that row. The simultaneous equations arising in many 
physical systems satisfy this condition. In these cases the iterative method 
of Gauss-Seidel can be applied for the solution of the unknowns. 

The mathematical derivation of the convergence criteria of the Gauss- 
Seidel iteration method for solving sets of simultaneous equations is 
complicated as the criterion of convergence is related to eigenvalues (see 
Chap. 2) of a matrix derived from the matrix of the coefficients, matrix 
A in the following discussion [7]. It will be seen in the succeeding 
chapter that the computation of eigenvalues for high-order matrices is a 
difficult numerical problem. For this reason, in practical cases, the 
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convergence criterion of Gauss-Seidel method can be applied only in the 
case of sets of simultaneous equations consisting of very few equations— 
Lor 5. 

Given the set of simultaneous equations 


@14%X1 + @joX%q + +++ + A,X, = b, 
G54 Bs + tigg ho eS OS eS 
21*1 2 2 
Bi*i 0659-4" 2 ke, = Oe 
or, in matrix notation, 
Ax = B (1.108) 
Equations (1.107) are solved for the diagonal unknowns 
: by (2 2 Q13 = A, : 
1 = e—_— — a ) + — 3 ee a n 
er ayy a41 Sit 
Z bo (= : A93 : ao, : 
9 ee ee oa + — 3 eS eC ee 
A959 Ao ao9 a99 (1.109) 
s 6, (= - und : a, n-l 
Se Se - eA 
ee ann ann Itt 
and in matrix notation 
ye Be Cx (1.110) 


where C is a matrix with diagonal elements of zero. Gauss-Seidel 
iterations start by assuming that Xo = X%g = +++ = x, = 0 and solving the 
first of Eqs. (1.109) for x, using this value of xy ithe Ng Sg ae 

= 0, solving the second of Eqs. (1.109) for X5, and so on. After 
ao the set of values for x, through x,,, the second iteration starts 
by using these values in Eqs. (1.109). In most practical cases solution is 
terminated where the values of the unknowns from one iteration to the 
next do not change within a specified number of significant figures. 
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Example. Solve the set of equations 


25x, =f 2X5 + Xe = 69 


2x4 + 10X59 + X%g = 63 (1.111) 


xX, + Xo + 4xXq = 43 


The solution starts by putting the above equation in the form (1.109) 
and carrying out the above outlined procedure. The values of x at the 
end of the first iteration are 


The solution will converge after five iterations to the values 


Computer Program for the 
Above Solution 


The following digital computer program in BASIC programming 
language was written for the Gauss-Seidel iteration method. Although 
most of the computer instructions are self-explanatory, the reader is 
referred to the BASIC programming language manual [2] for further 
explanation of instructions. 


100 DIM A(10,10),C(10, 10),B(10),X(10) 
110 READ N 

120. PRINT “COEFFICIENT” 
130° FOR l=1 TON 

140° FOR J=1T TON 

150 READ A\(1I,J) 

160 PRINT A(I,J); 

170 NEXT J 

180 PRINT 

190 NEXT | 

200 FOR I=1 TON 

210 FOR J=1 TON 

220 LET C(I,J)=A(I,J)/A(I,1) 
230 NEXT J 
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240 LET C(I,I)=0 

250 NEXT | 

260 PRINT “RIGHT SIDE” 
270 FOR I=1 TON 

280 READ B(I) 

290 PRINT B(I); 

300 NEXT! 

310 PRINT 

320 FOR I=1 TON 

330 LET X(I)=0 

340 NEXT | 

350 LET L=1 

360 PRINT “ITERATION VALUES OF X(1) THRU X(N)” 
370 FOR I=1TON 

380 LET D=0 

390 FOR J=1 TON 

400 LET D=D+C (I,J)*X(J) 
410 NEXT J 

420 LET X(1I)=(B(I)/A(I,I))—D 
430 NEXT | 

440 PRINT L; 

450 FOR l=1 TON 

460 PRINT X(I); 

470 NEXT I 

480 PRINT 

490 IF L=10 THEN 570 
500 LET L=L+1 

510 GOTO 370 

520 DATA3 

530 DATA 25,2,1 

540 DATA 2,10,1 

550 DATA 1,1,4 

560 DATA 69,63,43 

570 END 


Statement 100 of the program reserves computer memory spaces for 
the coefficients of Eq. (1.108). The matrix C of Eq. (1.110) is also 
shown in the dimension statement 100. Statement 110 reads the number 
of the simultaneous equations. Statements 130 through 190 read the 
coefficients of the unknowns of Eqs. (1.107). Statements 200 through 
250 compute the terms of the matrix C of Eq. (1.1 10). Note that the 
diagonal elements of this matrix are set to zero by statement 240. 
Statements 270 through 300 read the elements of the right side of Eq. 
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(1.107). Statements 370 to 430 compute the values of x’s at each 
iteration according to Eq. (1.110). Statements 450 through 470 print 
out the answers. Statement 490 limits the number of iterations to 10. 
The output of the program for 5 iterations of the above example problem 
is as follows: 


COEFFICIENT 

ae, 1 

2 14 

1 1 4 

RIGHT SIDE 

69 63 43 

ITERATION VALUES OF X[{1] THRU X[N] 
1 2.76 5.748 8.623 


2 1.95524 5.04665 8.99953 
3 1.99629 5.00079 9.00073 
4 1.99991 4.99995 9.00004 
5 Z. 2. 3. 


In this chapter we have discussed the introductory material on matrices 
and linear simultaneous equations. However, we have not treated the 
question of round-off errors in the computation and the stability of 
solutions. There are a number of texts which discuss these problems 
from a mathematical point of view [4, 7]. In most engineering appli- 
cations, however, these questions are studied by an examination of the 
answers obtained for a variety of input parameter conditions. For 
example, the simplest method of examining the correctness of the inverse 
matrix A™! is to multiply it by A and obtain the unit matrix. The 
question of the stability of solution, for example, for matrix inversion, 
can be answered by perturbing one or some of the matrix elements 
slightly and examining its effect on the solution. 


PROBLEMS 


1. Given the two equations 


ax + 3y = 0 
on ee — Ky =O 


for what value of a do these equations have a nonzero solution? 
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Find the value of the ratio x/y for each value of a. 
2. Expand the determinant 


| ee eee 
is 7a a 
ae ee, 


4. Verify that x = 2 is a root of the equation, 


sl = ee a 
6 (Sed oh esha, 26 
2 Ld (5 — x) 


5. Evaluate the determinant 


ia 4 22 a 

a See ee 

a2 aie 22-4 

ae ee eee | 

by the method of Chio. Answer: —1820 
6. Evaluate the determinant 

‘a ae 
Soe 


10. 


Li; 
i2. 
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by following the logic of the FORTRAN program given for determi- 


nant evaluation. Answer: 25. 
Show that 
2 cos 6 1 0 
pe 4 oe 1 | 54? 
sin @ 
0 1 2 cos 6 


provided that 9 is not a multiple of 7. 
Solve the equations 


9x — 17y + 2lz = 66 
inx: + 8y-—- Ta Se 
1X 2°39 92 = a 


Answer: x = 2.818, y = 3.821, z= 5.029. 
Find the values of a for which the equations 


Ce Be ee 
~Gs +(2— aly = 22 = 0 
4x = 29 —- (13 + @2.=— 0 
can be satisfied by values of x, y, z, not all zero. For each value of 
a, find the ratios x:y:z. 


Solve the set of equations for x,, x,, and x, by the method of 
elimination of Gauss 


1X4 + 2X5 + AX. = 9 
3X4 — 4x» + DX 2 = 44 
Answer: x, = 1, %5 = 3, X% = -1. 


Solve Prob. 10 by triangulation method. 
Show that the equations 
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13. 


14. 


15. 


16. 
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| 
3 


SES Ce SS 


| 


2x + oy + 4z = 0 


| 
—" 


ox + 4y + Oz = 


have a unique solution if c # 1 and find it. Discuss the case c= 1. 
Prove that 


«eee Bees 5 Se i ee 
ee b ee ee ae: 
| ee a . a. x q r p 


where p = ax + by + cz, q = az + bx + cy,r = ay + bz + cx. 
Verify that 


P= 8. - 3 
a See ae. 
= 
a 
eS a 
If 
+ 2 
2. ek 
A = SS a oe 
o.9..-4 
-] 2 


show that AB =O. This shows that from AB = 0 we cannot deduce 
that A=OorB=0. 


Bs 


18. 


19. 


20. 


Zi. 


———EE————E——EEE - ——- es ———————————— ————————— — oe 7 7 =r nn 
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Go =a 
Ac 

= 
show that A2 = 0, and hence that we cannot deduce from AP =H 
that A = 0. 


Show that the commutative law holds for the product of two 
matrices of the form 


If A and B are square matrices of the same order, show that, unless 
AB = BA, 


a (A. = BA = Be AO Re 
b. (A + B)? # A2 + 2AB + B? 
c. (A — 2B)(A — 3B) # A* — 5AB + 6B? 


If 


If 


See 
A= as oslk @ 
0 =i —-S 


show that A is singular. Construct adj A. 
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fap 


2D. 


24. 


25: 


26. 


2a: 


28. 
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If 


i) 
bo 
bo 
On 
| 
bo 
© 


A=|{2 5. 51 show that ALY = 21-2 ee 
| rs 


i) 
On 
oad 
asea 
oS 


where a = exp (27i/3), i = (-1)1”. 
If A is a square matrix of order n and if x is a scalar, show that 


|Ax| = |Alx™ and that adj (Ax) = (adj A)x”~ 1 


Obtain the inverse of the matrix 


ae. 
Pa) 2 
yr 


by the augmented matrix method. Check your solution by multi- 
plying the inverse matrix [A]~! by matrix [A]. 

Follow the logic of the FORTRAN program given for the aug- 
mented matrix method in inverting the matrix of Prob. 25. 

Verify Eqs. (1.100) and (1.101), which were used to obtain inverse 
matrices by matrix partitioning. 

Obtain the inverse of the M matrix 


29. 


30. 


at. 


Sas 


33, 
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oo eee 
Ms fo LS 
12 ade BA 


by partitioning the matrix and using Eqs. (1.101). 
Obtain the inverse of matrix of Prob. 25 by the matrix partition- 
ing method. 
Solve the set of simultaneous equations 
2x1 a 10x5 ate X93 ae 63 
X, + Xo + 4x3 = 43 


by the matrix inversion method. 
Solve the set of equations 


2x, - 8%) + Xg = —4 
=X 4+. ty FX = 9 

by the matrix inversion method. 
Solve the set of simultaneous equations 
4x, + Xo - X%g = 17 

X, + 6X9 - 2x, = 14 

X, + Xo - DX, = tf 

by the Gauss-Seidel method. 
Solve the set of equations 

3X4 + Kg + Xe = 4 
nD 


II 


X1 + 4X0 = hs 
xy =- Xo aS 6X3 = ie ys 


by the Gauss-Seidel method. 
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W EIGENVALUES, 
EIGENVECTORS, AND 
QUADRATIC FORMS 


The study of many physical systems of practical importance, such as 
the investigation of the elastic vibrations of a bridge or any other solid 
structure, the flutter of an airplane wing, the transient oscillations of an 
electric network, or the buckling of an elastic structure, leads to the 
solution of what is known in the mathematical literature as an eigenvalue 
problem. A general mathematical discussion of the eigenvalue problem 
and its ramifications is given in this chapter. This class of problems is 
also referred to as characteristic value problems. The terms “‘eigen”’ and 
‘‘characteristic’’ are used interchangeably. 


THE EIGENVECTOR (CHARACTERISTIC 
VECTOR) EQUATION 


Let us assume that we are given an nth order square matrix A and 
an nth-order column vector X. Then A and X are conformable matrices 
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and can be multiplied in the order AX to yield another vector Y, that is, 
8 Sore (2.1) 


In general, the vector Y will not have the same direction or magnitude as 
the vector X. Let us ask the question of whether or not it may happen 
that the vector Y has the same direction as the vector X. If this is the 
case, Y = AX where dA is a multiplicative scalar. If we place Y = XX in Eq. 
(2.1), we obtain 


AX = AX i222 


Equation (2.2), usually called the matrix eigenvector equation, arises in 
many branches of applied mathematics. The main problem is to find the 
values of A and the corresponding vector X for which Eq. (2.2) holds. 
Before considering the general case, let us consider the case of a second- 
order square matrix A given by 


A = and X = (233) 
eee | Xo 


If we expand the eigenvector equation (2.2) in this case we obtain 


| Sees x4 x4 
= A (2.4) 
ee Xo Xo 
If we multiply the matrix A by the column vector X as indicated in 
Eq. (2.4), we obtain the two scalar equations 


ax, > bX 5 = AX4 


(2.5) 
CX, + AXy = AXo 
Equations (2.5) can be rewritten in the form 
(d= A)x, + bx, = 0 
os (2.6) 


CX, + (d - A)Xo = 0 
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Equations (2.6) are a set of homogeneous equations of the type discussed 
in Chap. 1. In order for these equations to have nontrivial solutions for 
x, and x,, it is necessary that the determinant of the coefficients p (A) 
vanish, that is, 


Oak b 
C d=% 


pA) 


(2.7) 


| 


= det |A — Al| 

= 0 
The matrix K(\) = [A — Al] is called the characteristic matrix of the 
matrix A; the polynomial p(A) = det (A — Al) is called the characteristic 
polynomial. Equation (2.7) or p(A) = 0 is the characteristic equation 
of the matrix A. 
THE EIGENVALUES OF A 


The characteristic equation (2.7) of the matrix A has the form 


my ae ae = ee (2.8) 


The two roots A, and A, of this quadratic equation are the eigenvalues 
of the second-order matrix A given by Eq. (2.3). 

If A, and A, are the roots of Eq. (2.8), we can write the polynomial 
p(A) in the following factored form: 


ee 3 = 2 a, es 2 (2.9) 
If we now compare Eas. (2.8) and (2.9), we see that the coefficient 


of the —\ term (a + d) of Eq. (2.8) must be equal to the coefficient of the 
—\ term of Eq. (2.9), and we must therefore have 


Mp dg = (@ 4 @ = TrA-s Trace of A (2.10) 


The constant term of Eq. (2.8) must correspond to the constant term 
of Eq. (2.9); therefore, we must have 


A, Ag = (ad — be) = det A = determinant of A (2.11) 
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The important fact that the sum of the eigenvalues of a matrix A 
equals the sum of the principal diagonal elements of A or the trace of A 
and that the product of the eigenvalues of A equals the determinant of 
A is true not only for second-order matrices, but is true also in the 
general case of a matrix of order n. This fact will be established later. 
The two eigenvalues can be obtained by solving the quadratic equation 
(2.8) (the characteristic equation of A). 


THE EIGENVECTORS OF A 


The eigenvalues of A are the numbers A, and Aj. In order to obtain 
the eigenvectors of A that correspond to these eigenvalues, we return 
to the simultaneous equations (2.6). From these equations, we may 
obtain the ratio x./x, in the form 


Xx 
ee (2.12) 
x4 b A= d 
If we place A = A, in Eq. (2.12) we obtain the ratio 
A; - a 
ry = : Ga 


We thus obtain Xo = 11 X1- This fixes the ratio of X» to x, and therefore 
the direction of a certain vector. If we place A = Ay in Eq. (2.12) we 
obtain 


Ay - a 
b 


(2.14) 


We thus obtain x, = ryx,, and therefore another direction of a vector. 
We can write X, for a vector of undetermined length having the direction 
r;, and X, for a vector of undetermined length having the direction lo, 
and obtain 


(2.15) 
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The vector X, is then said to belong to the eigenvalue , and the vector 
X, to the adie Ag. Since the quantity x, is not specified, only 
the directions of these vectors are fixed. If we choose x, to be given by 


x, = —————— and xy 


1 
a.) Se ie — 


in X, and Xo, respectively, the vectors will then be normalized to have 
unit length. 

It is seen from the above discussion that the eigenvector equation 
gives eigenvalues and eigendirections. In most discussions involving eigen- 
values the vectors are normalized to unit length and in such case they are 
uniquely defined. 


THE EIGENVALUES AND EIGENVECTORS 
OF AN nTH-ORDER SQUARE MATRIX 


If we expand the eigenvector equation (2.2) in the general case in 
which the matrix A is a square matrix of the nth order, we have 


G41%X1 + Ayo%yo + +72 + M_% = AX 
Ao1X1 + AggXg + +++ + AgnX, = AXo (2.17) 
Q1X1 + Gn,9X%o + + @,,X, = Ax, 
This set of equations can be written in the form 
(a,, — AVX, + Qy9X% +++ + inXn = 0 
f21*1 + (Ao9 - Rig He Ay,X, = (2.18) 
a 4%, + G,glig 0M Ky = 0 
In matrix notation, Eq. (2.18) can be written in the compact form 
[A — AI](X) = 0 (2.19) 


where | is the unit matrix of order n. The homogeneous equations (2.18) 
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have a nonzero solution only in the case where the determinant p(A) of 
the coefficients vanishes. We therefore have 


p(d) = det [A - Al] = 0 (2.20) 
In matrix algebra, the matrix 
K(A) = [A —- Al] (2724) 


is called the characteristic matrix of the matrix A. The determinant of 
the characteristic matrix p(A) is the characteristic polynomial of A. This 
determinant is generally of order n and therefore is a polynomial of the 
nth degree in A. The equation p(A) = 0 is the characteristic equation of 
the matrix A. 


FUNDAMENTAL PROPERTIES OF THE 
CHARACTERISTIC POLYNOMIAL OF A 


If A is an nth order matrix the characteristic polynomial can also be 
written in the form p(A) = det[Al — A] = 0. In this form it results in 
the characteristic equation with ” coefficient of unity, 

p(A) = Me ae 8 eee = 0 (2.22) 
Setting A = 0 in Eq. (2.22) will result in 
p(0) = a, 2.25) 
However, since p(A) = det [AI — A], we have 
p(0) = det [-A] = (-1)"det[A] (2.24) 
If we substitute Eq. (2.23) mto Eq. (2.24), we obtain 
a, = (-1)" detA (2.25) 
If we assume that the matrix A has n distinct eigenvalues 


(Ay, Ag, --- AY), then since the eigenvalues are the roots of the charac- 
teristic equation, we can write Eq. (2.22) in the factored form 
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p{A) = (A - A,) A-= Ao) = ee x) (2.26) 

If we place A = 0 in Eq. (2.26), the result is 
p(Q) = (-D)"A, Ag Ag eee vA, (2.27) 


If we compare Eqs. (2.24) and (2.27), we have 


det {Al = Aya ey (2.28) 
Therefore the determinant of the nth order square matrix A is equal 
to the product of the eigenvalues of A. It follows that if det [A]=0 so 
that A is a singular matrix, then at least one of its eigenvalues is zero. 
If the factored form of p(A) of Eq. (2.26) is multiplied out, it will be 
found that the coefficient of \”~! which is a, in Eq. (2.22) is given by 


Ay + Ag + Ag tees tA = =@; (2.29) 


If, on the other hand, the determinant, det [A - XJ], is expanded to 
determine the coefficient of a, or \”~ ! in the polynomial p(d), the 
result is 
ay = Ady 2-655 6 8, 4-4 = “Teaee A (2.30) 
since by definition the trace of the matrix A, TrA, is the sum of the 
principal diagonal terms of A. If Eqs. (2.29) and (2.30) are compared, it 
is seen that we have the important relation 
Trace A = Ay + Ag + Ag teres +A, (Fist) 
The above equations can be used primarily as a check on computation 


of the eigenvalues. The nth-order polynomial of Eq. (2.22) can be solved 
for characteristic values by the methods outlined in Chap. 3. 


NEWTON’S FORMULAS FOR THE COEFFICIENTS 
OF THE CHARACTERISTIC POLYNOMIAL 


We have seen that the coefficient a, of the characteristic polynomial 
p(r) is given by the relation a, = detA and that the coefficient 
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a, =—Trace A. The mathematician M. Bocher has given a set of equations 
for the coefficients of the characteristic polynomial p(A).* 
We define the numbers S,, S,, Sz, ..., S, by the equations 


S, = Tet4) S, = THIAD -S, = TRA... S> = TIA) O37) 


so that S, is the trace of the kth power of the given nth-order square 
matrix A. Bocher has shown by a long algebraic argument based on 
Newton’s formulas that the coefficients a, ad, ..., a, of the character- 
istic polynomial p(A) of A can be computed successively by the following 
recurrence formulas: 


a, 2 —S, 
ay Sy + So 
" 
. 2 


As a numerical example of the use of these formulas, let it be required to 
determine the characteristic polynomial and therefore the characteristic 
equation of the matrix 


ee ee 
Aa y+ 9. 4-5 (2.34) 
-8 O -3 


We first compute A? and A? and obtain 


*M. Bocher, “Introduction to Higher Algebra,” pp. 243-244, Dover Publications, 
Inc., New York, 1964. 
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ee ee se =e 48 
jae ee eee eee 24 = 2 (2.35) 
eee ee: ig 4} 


We then compute the traces 


S02 A = 3 2 ee aes 


(2.36) 
By the use of Eq. (2.33) we then obtain 
a, = -S, = -3 
ay Sy = So 
a, = —-—————_ = ek 
2 Cai 
AS, + 2155 + Ss 
as a 
3 
The characteristic equation of the matrix (2.34) is, therefore, 
Via DP eS SD (2.38) 


Equation (2.25) in this case gives detA = —3. 

It should be noted that although Bocher’s formulas represent an 
elegant mathematical relation, in practical cases they are not used in 
computation of the coefficients of the characteristic polynomial of high 
order (3d and higher) matrices. The reason for this is that raising 
matrices to various powers, in addition to requiring a great number of 
calculations, in many cases results in loss of accuracy as the elements of 
the new matrices require more significant figures. The following numeri- 
cal method, due to A. N. Krylov, can be used for the determination of 
the coefficients of the characteristic equation. 


THE METHOD OF A. N. KRYLOV FOR 
DETERMINATION OF THE CHARAC- 
TERISTIC EQUATION 


Consider the matrix 
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vg, Be Fees An 
a1 an2 2n3--°° Inn 


whose characteristic equation is desired. As discussed before, the char- 
acteristic equation results from setting the determinant 


ai4- A A419 a iy 
a | ne ee 

DA) = = 22 2n (2.40) 
a4 a9 es Gee 


equal to zero. The method of A. N. Krylov consists of reducing the 
determinant of Eq. (2.40) to the form 


Oe ed Pere 


eee eee ee 
BOY = a 22 2n | _ 9 (2.41) 


ee ge ee 


where the characteristic value X appears only in the first column. 
Expanding the above determinant in terms of its first column will give 
the characteristic equation in powers of A. Note that the form of Eq. 
(2.41) is such that the characteristic polynomial can more readily be 
obtained than from the expansion of the determinant of Eq. (2.40). 

To derive the above relation we start by noting that Eq. (2.40) is the 
necessary and sufficient condition for the system of homogeneous 
equations 


AX, = G1 %Xy + AyoXq + +++ + A,X, 
AXo = A511 + A959 Xo + gg es + Gon*n 
| (2.42) 


AX =O _aky + Goke t+ + a x 


Eigenvalues, Eigenvectors, and Quadratic Forms 59 


to have a solution x,, x5, X3,..., x, different from zero. Multiplying the 
first equation by ) and replacing the right side values of Ax, Axo, AXg, ... 
of the resulting equation by their equivalents from Eq. (2.42), we get 


a; = bo, X41 = boo Xo SO es bo Xn (2.43) 
where 
bop ee > 115% sp k = 1, 2s 
s=] 


Multiplying Eq. (2.43) again by 4 and replacing the right side of the 
resulting equation by equivalent values from Eq. (2.42), we get 


3 %y = bg,%1 + bggX_ + --- + 03,,%,, (2.44) 
Repeating the above process will result in the set of equations 


AX, = by, %1 + DyQ%_ + +--+ + 2, x, 


2 
r X41 = 051 X41 es Doo Xo eb ia a bo, X, 


(2.45) 
AY xy = b1%4 * b,9%2 ar te as Oan*n 
where 
bi, = 4, 
; y : 1 = 2, nt 
: = << a 
. s=1 oe eee ees 


The set of equations (2.45) will have a nonzero solution in x’s only if 
the condition on the determinant of Eq. (2.41) is satisfied. 
Expanding the determinant of Eq. (2.41) about the first column and 


denoting the cofactors of element (7, j) by C, j> we get 
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n n-1l n-2 
Chia wi ae +C,_oA + 25 


; (2.46) 
e04 44c : biC, = 9 
=I 
Dividing through by the coefficient of the \” term, we get 
ee ey ee ee ee a |. (2.47) 
where 
C. 
P; = = J = i= Si ] 
Bay 
n, 
CG 
il 
i ee =>) aS 
i=1 n,1 


In Eq. (2.47) the p’s are the coefficients of the characteristic equation of 
the matrix A of Eq. (2.39). Having obtained this equation the roots can 
be evaluated by root finding methods of Chap. 3. It should be noted that 
in many physical problems a great deal of information can be extracted 
from the characteristic equation without actually solving for the roots. 

Example. It is required to find the characteristic equation of the 
determinant 


2 4 -6 
A=| 4 2 —6 
-6 -6 -15 


From Eqs. (2.45) the values of the coefficients b;, can be calculated 
Ob, = 4, = 2 dbyg = yg = 4 yg = G43 = -6 


and 
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3 
ie » bie %51 = 411411 + 512491 + F13%31 = 56 
s=1 


Similarly 
bs, = -4 bgp = 4 bag = —1,458 


Having these values, the determinant corresponding to Eq. (2.41) can be 
written 


2-2 4 ~6 
D(A) =| 56-A% 82 54 
Se eS ee sae 


The cofactors of elements (1, 1), (2, 1), and (3, 1) can be calculated as 
follows 


C,, = (52)(-1,458) - (54)(4) = -76,032 
Cy, = —((4)(-1,458) - (-6)(4)) = 5,808 


C3, = (4)(54) - (-6)(52) = 528 


The summation term of Eq. (2.46) can be calculated from the above 
values 


1) bi Cj, = — (by, Cy, + 59101 + 531031) 
i=l 


= —171,072 
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Using the calculated values in Eq. (2.46), we get 
528A° + 5,808A7 — 76,032A - 171,072 = 0 
After dividing by the coefficient of A? term, we have 
We 4-T1A4. = 1444 = 324 = 0 
As mentioned before, the roots of this equation can be obtained by 
root-finding methods of Chap. 3. The characteristic vectors corresponding 


to these roots are obtained by solving the set of simultaneous equations 
(2.42). 


DETERMINATION OF THE EIGENVECTORS OF A 


As we have seen, the eigenvector equation AX = AX can be written 
in the form 


[A — AI](X) = K(A)(X) = 0 (2.48) 
where K (A) is the characteristic matrix of A. 
To each eigenvalue A = A,, there corresponds a column vector X, 


called an eigenvector. The eigenvector X, is said to belong to the eigen- 
value A ,; it satisfies the equation 


eet = 0 PS eee (2.49) 
Equation (2.49) gives nonzero solutions for the numbers in the column 
X. since detK(A;) = 0. 

As a consequence of the definition of inverse matrix, we have 
K(A) adj K(A) = det K(A)I = pi)! (2.50) 
If we place A = A, in Eq. (2.50), we have 


K(A,) adj K(A,) = pid) = 0 (2.51) 


This result follows from the fact that p(A,) = 0, since A, is an eigen- 
value of A. If we now compare Eqs. (2.49) and (2.51), we conclude that 


Eigenvalues, Eigenvectors, and Quadratic Forms 63 


X, can be taken to be proportional to any row of adjK (\,); we can then 
write 


X. = (const)(any column of adj K (A,)) (2.52) 


By the use of Eq. (2.52) we again obtain an eigendirection, as was the 
case in the simple situation of the previously given second-order square 
matrix. 


a 
In that simple case, we had A = 


b 
- in this case we have 
ager | 


aA b 
K(A) = (2.53) 
e d -— x 
We now take the transpose of K (A) and obtain 
: a ec 
K (A) = (2.54) 
b ge 
The adjoint of K (X) is therefore 
dX 6 
adj K(A) = (2.55) 
—C f= Kx 


Then by the use of Eq. (2.52) we have 


d= x —b 
X, = (const) | or X, = (const) (2.56) 
—C a-—aA 


Similarly, the eigenvector X, is proportional to either one of the columns 
of adjK(A,). This is the same result that we obtained previously. 

The result (2.52) is true in general and enables one to obtain the 
eigenvectors of a matrix whose eigenvalues are distinct. It sometimes 


64 Digital Computer Methods in Engineering 


happens that the characteristic equation (2.22) has repeated roots. In 
this case the matrix A is said to have multiple eigenvalues. We shall 
reserve the discussion of multiple eigenvalues for a later section. For 
the moment, we will consider the case for which the characteristic 
equation (2.22) has distinct roots and therefore the matrix A has n 
distinct eigenvalues A,, Ag, Ag, ---, A,,- In such case the eigenvectors or 
eigendirections are given by Eq. (2.52). 


PROPERTIES OF THE EIGENVECTORS OF A 
SQUARE MATRIX 


In this section we shall discuss several of the most important properties 
of the eigenvectors and eigenvalues of a square matrix A. These properties 
are of great theoretical and practical importance. 


Property 1. Let A be a symmetrical matrix, so that a;; = 4;; OF 


A =A. If the eigenvectors X, and X ; belong to the eigenvalues A, and 
A, , where A, 7 Ai then X, and X j are orthogonal vectors. That is, their 
scalar product vanishes, or 


oe ge X.X, 2 G57) 


AX. = XX. (2.58) 


If we take the transpose of Eq. (2.58) and use the reversal law of | 
transposed products, we obtain 


| oF ee oa (2.59) 


l l l 
If we postmultiply Eq. (2.59) by X 5 we obtain 
XjAX, = A;X;X, (2.60) 


The vector Xx, satisfies the eigenvector equation 


AX, = A,X; (2.61) 
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If we now premultiply Eq. (2.61) by X, we obtain 

Xx, AX, - A,X; X, | (2.62) 
If we now subtract Eq. (2.62) from Eq. (2.60), we have 

0 = (A; - A)X;X, (2.63) 
Since aA. # dis Eq. (2.63) gives the important relation 

XX, = 0 (2.64) 
Since the scalar product vanishes, X, and X, are orthogonal. 

Property 2. If A is a real symmetrical matrix (a matrix all of whose 
elements are real numbers), all of its eigenvalues are real. This is not an 
obvious fact since the roots of an nth-order equation with real coeffi- 
cients are, in general, complex numbers. 

Let X be a typical eigenvector of A; therefore, we have 

AX = )X (2.65) 

The complex conjugate of this equation is 


AX* = d*x* (2.66) 


where the symbol * denotes the complex conjugate. The matrix A, 
being real, is unchanged. The transpose of Eq. (2.66) is 


ar ae ey add (2.67) 
We now postmultiply Eq. (2.67) by X to obtain 

AX = (2.68) 
But if we premultiply Eq. (2.65) by X*° we have 


X*’AX = dX**X (2.69) 
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If we now subtract Eq. (2.69) from Eq. (2.68) we have 
sO eer (2.70) 


However, if X consists of the elements x,, x5, x5, ..., x,, then X*’X is 
the sum of the squares of the absolute values of the elements of X. 
Hence X*’X is a nonzero positive quantity, and therefore to satisfy Eq. 
(2.70) we must have 


A* =A (2.71) 


This equation implies that » is real. Since A is a typical eigenvalue, all the 
eigenvalues of A are real numbers. 

Property 3. If A is a real, skew-symmetrical matrix, so that A’ = —A, 
all its eigenvalues are imaginary. This may be established by the same 
argument that was used to establish property 2. 

Property 4. The eigenvalues of a real orthogonal matrix are all of 
unit modulus (have absolute value of unity). An orthogonal matrix has 
the property that AA = I, that is, the inverse of an orthogonal matrix 
is its transpose. Matrices of this type are of great importance in dynam- 
ics and other branches of applied mathematics. Let A be an orthogonal 
matrix; its eigenvector equation is 


AX = AX (2.72) 


Although A has real elements, X and X are not necessarily real. The 
conjugate of this equation is 


AX* = A*x* (2.73) 
where A* denotes the conjugate of \. The transpose of Eq. (2.73) is 

AOA = NEXT (2.74) 
If we now postmultiply Eq. (2.74) by AX = XX, we obtain 


x AA = NAS (2.75) 
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Since A is an orthogonal matrix AA = I, Eq. (2.75) becomes 
MR eB (2.76) 
If we now divide by the scalar X*°X , we obtain 
MA = 1 (2.77) 


It is therefore evident that the modulus (absolute value) of A is unity. 
Since \ is a typical eigenvalue of the orthogonal matrix A, all its eigen- 
values are of unit modulus. 

The Transform of a Square Matrix (Similarity Transformations). In 
many applications of matrix algebra and matrix calculus to problems in 
physics and engineering, it is important to consider the matrix transfor- 
mation 


C28 a6 (2.78) 


where B is a nonsingular matrix. Such a transformation is called a 
similarity transformation. The matrix C is said to be the transform of A 
by the matrix B. It is interesting to note that if A is a symmetric matrix, 
(A’ = A) and B is an orthogonal matrix (B~! = B’), then the transform 
o1-A.-C is 


C = BAB (2.79) 
In this case we have by the reversal law of transposed products: 
C’ = (BAB) = BAB = C (2.80) 
Therefore, since C’ = C,C is a symmetrical matrix. 
Property 5. The eigenvalues of C = B~'AB are equal to the eigen- 
values of A. That is, the eigenvalues of the transform matrix C are equal 
to the eigenvalues of A. To prove this proposition, we note that the 


characteristic matrix of C is given by 


a LAB ADs ac BoA DB (2.81) 
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Therefore the characteristic polynomial of C, Q(A) is given by 


Q(x) = det (C — AI) = det B-!(det (A — AI) det B (2.82) 


since (det B~!) = 1/det (B), it is apparent from Eq. (2.82) that 


Q(A) = det (A — AI) = pir) (2.83) 


where p(A) is the characteristic polynomial of A. Thus, it is evident that 
the matrices A and C have the same characteristic equations 


Q(A) = p(dA) = 0 (2.84) 


Therefore, the eigenvalues of C are equal to the eigenvalues of A. 

Property 6. If C is the transform of A so that C = B'!AB, the 
determinant of C is equal to the determinant of A and the trace of C is 
equal to the trace of A. 

To prove this, it is only necessary to recall that the eigenvalues of A 
and C are the same. Since the determinant of a square matrix is equal 
to the product of its eigenvalues and the trace of a square matrix is 
equal to the sum of its eigenvalues, it is evident that A and C have the 
same determinant and the same trace. 

Property 7. The reciprocals of the eigenvalues of A are the eigenvalues 
of A! and the eigenvectors of A are the eigenvectors of Al. To prove 
this statement, consider the eigenvector equation of A 


AX = AX (2.85) 
Let us premultiply Eq. (2.85) by A7! to obtain 
Was Ik ant FX (2.86) 
This equation can be written in the form 
ALN NAR (2.87) 
Equation (2.87) is the eigenvector equation of A7l: therefore, 47! is the 


eigenvalue of A™! and X is the eigenvector of A. Since » and X are 
typical eigenvalues and eigenvectors, it is easy to see that the reciprocals 
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of the eigenvalues of A are the eigenvalues of A!, and that the eigen- 
vectors of A are the eigenvectors of A”. 


THE CASE OF REPEATED EIGENVALUES 


In certain practical cases it happens that the characteristic equation 
(2.22) has multiple roots; in such a case the matrix A is said to have 
multiple or repeated eigenvalues. The question of determining the 
eigenvectors that belong to a repeated eigenvalue is a difficult one. We 
shall consider here only the simplest aspects of this question. In order 
to make matters definite, we shall consider only a third-order case and 
then make the proper generalizations to cases of higher order. 

Let A be a (3, 3) matrix with a triple eigenvalue A). Let X,, Xo, and 
X, be the eigenvectors that belong to the eigenvalue Ao: The eigenvector 
equation for the eigenvalue 4, is written as 


Ae = 

(2.88) 
Mee 
AX, = Neko 


We can multiply the first of the equations (2.88) by a, the second one 
by b, and the third one by c, add the results, and thus obtain 


A[ax, + bX5 + cX4] = Ag lax, + bX5 + cXg] (2.89) 
If we now let 

Y = (aX, + bX_ + cX3) (2.90) 
it is apparent that Eq. (2.89) can be written in the form 

AX == MY (2.91) 
Therefore, Y is an eigenvector of A belonging to the eigenvalue A,; hence 
a linear combination of the eigenvectors belonging to the multiple eigen- 


value X, is also an eigenvector belonging to A,. This fact can be utilized 
in obtaining eigenvectors belonging to multiple eigenvalues. 
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Example. As an example of a symmetrical matrix that has a double 
eigenvalue, consider the matrix 


.  ° 1 
Age A Ad (2.92) 
, Dials ieee | 
This matrix has the following eigenvalues: 
eee hy = ee (2.93) 
An eigenvector belonging to 4, = —2 can be taken to be 
a 
X, =| 1 (2.94) 
i 
The eigenvector equation AX 9 = AgXq when written out is 
t HE Fee Fe | Xo (2.95) 
ges wee) Xe Xg 


This equation if expanded yields three equations of the type 
X3 aS (x4 = Xo) 


(2.96) 
We can therefore take 


QI) 


for the eigenvector belonging to the eigenvalue Ay = 1 


Eigenvalues, Eigenvectors, and Quadratic Forms 71 


In order to obtain the eigenvector X, belonging to A, = 1 we note 
that the matrix A is a symmetrical matrix and that as a consequence of 
property 1 above, the eigenvectors of A are mutually orthogonal. Let us 
make use of Eq. (2.96) in the form 


a (2.98) 


and take x» = h and x, = k so that the third eigenvector has the form 


ae (2.99) 


We now adjust X, so that 


Xi Xg = -h+h)+h+k=0 (2.100) 


X,Xz Shs ah or k= SR hs BP Se Sh 


If we substitute these values of k into Eq. (2.99), we obtain 


= ft = 
ee h\|= hj] 1 h = an arbitrary constant (2.102) 
pA 2 


<1 1 1 
eo ee See oe Oe (2.103) 
1 0 2 


as a set of three orthogonal eigenvectors of the matrix A. 
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Example. Let it be required to find the eigenvalues and the eigen- 
vectors of the matrix 


7 4 -4 
wet 4.8 4 (2.104) 
-4 -1 -8 


The eigenvector X, can be found by the standard procedure of Eq. 
[z.3,} to be 


eee ae (2.106) 


When A, = Ag = -9, the eigenvector equation will be found to give three 
equations all proportional to the equation 


iy + Xo - Xg = 0 or 4h, = X_ — Xo (2.107) 


If we choose Xo = 1, x3 = 1, then x, = 0; therefore, we can take Xo to be 
0 

Ae = (2.108) 
1 


If we now take Xo = h, xg = k, thenx, = (k - h)/4, and we can therefore 
take 
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4 (2.109) 


Since A is symmetric, X,, Xo, and X. are orthogonal, therefore Le Sgt 
0, X,X,=0. We find that k = —A ensures orthogonality, and if we 
choose k = 2, we have 


Ka =) =2 (2.110) 


4 0 a 
fee Aaah ee ee preg Ny Se (2.111) 
Za 1 2 


MATRIX ITERATION METHOD OF 
EVALUATING EIGENVALUES AND 
EIGENVECTORS 
As discussed above, the characteristic values and vectors of matrix [A] 
are related as follows: 


[A - AI](S) = 0 (2.412) 


This matrix equation can be written 
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Qj, -A ayo Q13 QXy Sy 0 
G9) 9 gh Be (2.113) 
an1 a2 an3 aan me Sn 0 


where 2 is the characteristic value, a’s are the elements of matrix [A], 
and s’s are the elements of the characteristic vector corresponding to A. 
It is further noted that in many engineering applications the matrix A 
is a real-valued symmetrical matrix, that is, a,. = a., and a,.’s are real. 
From matrix theory discussed above it is known that the characteristic 
values of real symmetrical matrices are real. This property will be 
utilized in obtaining characteristic values and vectors by the following 
iteration process. 
Equation (2.113) can also be written in the form 


Gay gg Gag Ain Sy Sy 
Go, Gog gg +--+ Ag, So oan So (2.114) 
oy an2 a3 aan Sn Sh 


In the matrix iteration method of finding the value of A and the corre- 
sponding values of sj, S5,..-.,8, Wwe start with an arbitrary set of 
$1, S9,---, §,, and for convenience in calculations choose $ =i 
should be noted that Eq. (2.113) forms a set of homogeneous equations, 
1.€., equations involving s’s with the right side equal to zero. The 
absolute values of s’s cannot be determined. However, ratios of s’s can 
be obtained. By setting s_ = 1,in effect we are calculating the ratios of s’s 
O98 Ol 6118, Go/S,, Ba) Sin s_,/s,-) Setting these values of s’s on the 
left of Eq. (2.113) and multiplying by the matrix [A], we obtain a 
column matrix. Dividing this column matrix by the element in the last 
row s_, we obtain the next approximation to the value of A(= s,) and 
the characteristic vector consisting of s), s5,..., s,. Continuing this 
process the iteration will converge, resulting in the characteristic value 
and the corresponding characteristic vector [4]. It can be shown that 
the characteristic value obtained by this method is the largest characteristic 
value or equivalently the largest root of the characteristic equation. 
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It can be shown that the rate of convergence of this iteration process 
depends on the numerical separation of the eigenvalues of matrix A. 

Example. As a numerical example of evaluating characteristic values 
and vectors by the matrix iteration method, consider the following 
matrix: 


eS 
ee (2.115) 
se aS 


As a first approximation to s’s or the characteristic vector, consider the 
column matrix 


=e | (2.116) 


Note that the last element is set to unity. It is also generally known that 
the elements of the characteristic vector corresponding to the highest 
characteristic value are not all of the same sign. For this reason the sign 
of the center element is chosen as minus. The iteration method will, of 
course, converge even if [1, 1,1] were chosen. However, in this case 
more iterations will be required. Multiplying the two matrices and 
keeping the last element of the resulting column matrix as unity, we get 


et ee 1 1.5 
eS Se = IP ee ay 8S 
ioe yee | 1 1 


Using the column matrix on the right as the next approximation and 
following the above procedure, we get 
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ei 0 15 1.6666 
=e! Sees FS = 8 — 2.1666 
| ; 1 1 
1.6666 1.7368 
(2.118) 
same as above —2.1666 | = 3.1666] —2.2105 
1 1 
1.7368 1.7701 
same as above —2.2105 | = 3.2105] -2.2295 
1 1 
Continuing this process, iterations 10 and 11 will result in 
1.802 1.802 
3.247|-2.247] and 3.247 ]|-2.247 (2.119) 
1.0 1.0 


It is seen that iterations converge and the values at iterations 10 or 11 
represent the largest characteristic value (3.247) and the corresponding 
vector 1.802, —2.247, 1.0. It is sometimes desirable to normalize the 
characteristic vector to a value of unity. If $1, 85, and sq are the 
components of the characteristic vector, this results: 


re + Te - $5" = l (2.120) 


Or in the case of the above example, all the components should be 
divided by the square root of 


(1.802)% + (2.247)" + (1.0)? = 9.2962 (23945 


or 3.049. The normalized characteristic vector then becomes 
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— 0.591 
3.049 
2B ee (2.122) 
3.049 
2 | | ieee 
3.049 


FORTRAN PROGRAM FOR EVALUATION 
OF CHARACTERISTIC VALUES AND 
VECTORS 


The following FORTRAN program [1] has been written for the 
iteration process discussed above. The data entered in the program are 
the same as the above example. The output of the program consists of 
20 iterations and is given following the listing of the program. 

Statement 10 reserves computer storage for indexed variables. 
Statements 20 and 30 read the order of the matrix (N = 3) and 
the values of its elements S(J,J). Statement 40 reads the initial 
characteristic vector P(J) as (1, —l, 1). K in statement 50 is the 
number of the iteration and is indexed by one at each iteration. 
Statements 70 through 110 multiply the matrix by the initial char- 
acteristic vector. Statements 120 through 140 change the desig- 
nation of the characteristic vector from Z(I) to P(J) to be used in 
future iterations. Statements 150 through 180 print out the iteration 
number and the characteristic value which is equal to P(N). State- 
ments 190 through 210 divide the characteristic vector by P(N), 
as was done in the example problem, to keep the last element of 
this vector equal to unity. Statements 220 and 230 print out the 
characteristic vector. In statement 240 the number of iterations is 
arbitrarily set at 20. If K is less than or equal to 20, the loop starting 
with statement 60 is repeated and another iteration is obtained. 
The computer results are shown following the program. It is seen 
that the characteristic vector and the corresponding characteristic 
value to three significant figures remain unchanged after iteration 
number 10. 
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C PROGRAM TO EVALUATE CHARACTERISTIC VALUE AND VECTOR 
10 DIMENSION S(100,100),P(100),Z(100) 


20 DATA N/3/ 


30 DATA (5012) J=1,3) 1-1, 32.-1,,0.-1,,2.,7 68-1 


40 DATA (P(J),J=1,3)/1.,-1.,1./ 


50 K=0 

60 K=K+1 

70 DO 110 Il=1,N 
80 Z (I)=0. 

90 DO 110J=1,N 
100 Z(I)=S(1,J)*P(J)+Z(1) 
110 CONTINUE 

120 DO 140 I=1,N 
130 P(I)=Z(I) 

140 CONTINUE 

150 WRITE (2,160) K 


160 FORMAT (18H ITERATION NUMBER= 15) 


170 WRITE (2,180) P(N) 


180 FORMAT (22H CHARACTERISTIC VALUE= 1PF 10.3) 


190 DO 210 I1=1, N 
200 P(I)=P(I)/P(N) 
210 CONTINUE 


220 WRITE (2,230) (P(I), 1=1,3) 


230 FORMAT (22H CHARACTERISTIC VECTOR /(6X1PE10.3)) 


240 IF (K-20) 60,250,250 
250 STOP 
260 END 


Computer solution obtained: 


ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.500E 00 
—2.000E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.667E 00 
“2.41 16-00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 


2.000E 00 


3.000E 00 


3 
3.167E 00 


1.737E 00 
—2.211E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.770E 00 
—2.230E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.787E 00 
—2.239E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.795E 00 
—2.243E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.798E 00 
—2.245E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.800E 00 
—2.246E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.801E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
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3.211E 00 


3.230E 00 


3.239E 00 


3.243E 00 


3.245E 00 


3.246E 00 


3.247E 00 


79 


80 Digital Computer Methods in Engineering 


ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 


11 


12 


13 


14 


16 


16 


17 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 
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1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 19 
CHARACTERISTIC VALUE = 3.247E 00 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 20 
CHARACTERISTIC VALUE = 3.247E 00 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 


ITERATION METHOD FOR OBTAINING 
THE SMALLEST CHARACTERISTIC 
VALUES AND VECTORS 


The method described above gives the highest characteristic value and 
the corresponding characteristic vector. However, in many applications 
the lowest characteristic value and the corresponding vector are desired. 
To obtain this, we divide Eq. (2.114) by A and premultiply both sides by 
matrix [a he Denoting 1/A = w, we get 


| i a An "ys 
ee et ee ee (2.123) 

Sy an a2 an3 ann Sn 
We note that the iteration process for obtaining w and $j, So, S3,---, S, 


remains the same as before with matrix la; ] replaced by its inverse 
[a, J" as given by Eq. (2.123). From this equation the value of o will 
correspond to the largest characteristic value of the inverse matrix [a, J" 
or the smallest characteristic value of la, jl since wm = l/d. 

Example. The inverse of the matrix of the previous example can be 
obtained, for example by augmented matrix method, as written below 
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5 4 Gy Ss Se 
= er ae = oS Se 
ee eee toaegraes 


Using this inverse together with an initial iteration characteristic vector 


and proceeding with the iteration processs as was done in the above 
example, we get 


First iteration 


1 1 : 1 0.50 

1 2 2 1 = 6 0.8333 

+2 33 1 1.0 
Fifth iteration 

1 1 1 0.4451 0.4451 

a 0.8020] = 5.049] 0.8020 

+ 8 Sets 11.0 
Tenth iteration 

1 1 + 0.4450 0.4450 


L ..2 2) FOS@0197 = 5.049) 0:3018 
be ee ae 1.0 
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From this, the smallest characteristic value will be A = 1/5.049 = 
0.198 with the corresponding components of the characteristic vector 
0.445, 0.8019, and 1.0. 

This vector can be put in normalized form by dividing the components 
by the square root of the sum of the squares. This results in the values 
0.328, 0.591, and 0.736 (normalized) with the lowest characteristic value 
of A= 0.198. 


USE OF ORTHOGONALITY RELATIONS IN 
OBTAINING CHARACTERISTIC VALUES 
AND VECTORS 


After obtaining the largest characteristic value and the corresponding 
characteristic vector it is possible to use the orthogonality relations to 
obtain other characteristic values and vectors. (This method is also used 
when the characteristic equation has double roots, i.e., two characteristic 
values are identical.) From matrix theory we have that the characteristic 
vectors of real symmetric matrices are orthogonal. That is, if 
$75 86, 425 Bae the components of the characteristic vector corre- 
sponding to the characteristic value X, and sj, 85, 83,---, 8, are the 


n 
components of the characteristic vector corresponding to ’, then 


$18] + S989 + ++: + 8,8, = 9 (2.124) 


Assuming that we have obtained the largest characteristic value 4, and the 


corresponding vector sj, So, ..., s,,we can obtain the next characteristic 
value and vector \’and sj, s5, ..-, 87, by writing Eqs. (2.114) in the form 
a Fie Page 1 4 
ea fees 2a er 3 ae 
ee i gee ries Az» 
ad an2 ann s, s, 


ee ee ee (2.126) 
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Writing matrix equation (2.125) in the form of linear equations and 
substituting the value of s’ from Eq. (2.126), we get a set of n equations 
in(n — 1) unknowns sj, 85, .--, 87 _ 1. 

Since the last equation is redundant (it will always be satisfied) we can 
drop this equation and write 


biy bi9 Oy ant Ss} S} 
boy boo Sg 89 J S95 (2.127) 
fee ee Pee ee eee By ot 


where b’s are the new coefficients. These equations can be solved by 
the iteration process, discussed previously, for \’ and s}, s5,---, 8) _ 1. 
Having these values, Eq. (2.126) will give the value of s°. 

Additional characteristic values and vectors can be obtained by writing 
the orthogonality relations between the vectors already calculated and 
vectors whose values are desired, and using these relations to reduce the 
number of equations. 

Example. Consider the previously given example problem where the 
largest characteristic value and the corresponding vector of the matrix 


ee Meee DE 
"5 eae ee 
02-14 


were found by iteration method. These values are 3.247 and 1.802, 
—2.247, 1.0. It is desired, using the orthogonality condition, to find the 
next largest characteristic value A’ and the corresponding vector with 
components sj, sq and sq. 

The orthogonality condition of Eq. (2.126) results in 


(2.128) 


= -1.802s7 + 2.24785 


Eigenvalues, Eigenvectors, and Quadratic Forms 85 


The values of sj, s5,and s3 should also satisfy Eq. (2.125): 


+e | 0 S43 Si 
ae | ee So | = A’|S5 (2.129) 
(ee | r 83 83 


Writing the above relation in equation form and substituting for $4 in 
terms of sj and s¢ in the first two of these equations, we get 


ia = N) 84 - 85 = 0 
0.8028; + (-0.247 - A’) s = 0 (2.130) 


se Ne SD 


The last equation of (2.130) is not needed as the first two equations are 
sufficient to solve for s}, s5,and A’. Putting these in matrix form, we get 


ts ay, Si $4 
mage (2.131) 
0.802 —0.247 $5 85 


The above relations, which correspond to Eq. (2.127), can be solved 
by matrix iteration,* discussed previously, by assuming that s} = 1.0 
and s5 = 1.0 for the first iteration. The iteration process will converge 
to three significant figures at the sixth iteration, resulting in 


NV = 1.555 
S| = 2.247 (2.132) 
Sg = 1.0 


Using the above values in the expression for $3 in terms of Sj and $5, 
we get 


*Since in this case only two equations are involved, they can also be solved 
simultaneously. 
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$3 = — 1.8028; +. 2.24785 
—1.802 x 2.247 + 2.247 x 1.0 
— 1.802 


(2,433) 


The value of s, can be set equal to 1.0, as was done in the case of highest 
characteristic value and vector, by dividing the components of the new 
characteristic vector by —1.802. This results in 


, 2.247 


— Bs oe 

~ 1.802 
a eo = ~0.555 (2.134) 
ee SEM 

~1.802 


with A’ = 1.555. The above values can be normalized by dividing each 
component by the square root of the sum of the squares. This results in 


$4 = —().736 fa-2 —0.328 $5 0.591 


In order to calculate the next characteristic value and the corresponding 
vector we can proceed as follows. Assume the desired values are \” and 
Sj, $5, and s3. Write the orthogonality relations between these and the 
vectors S and Sas given in Eq. (2.124). 


1.802s; - 2.24785 + s3 = 0 
tf 4? Ld (2.135) 
—1.247s; ae 0.55585 a5 S9 = () 
Solving these equations for s3 and s¢ in terms of s7, we get 
s5 = 1.80287 
(2:136) 


S35 = 2.24787; 


Substituting the above relations for the values of sj and s3 in the first 


Eigenvalues, Eigenvectors, and Quadratic Forms 87 


of Eqs. (2.129) written for s”, 


y Sens ee S$] Si 
a ee ee 85 |= Also (2.137) 
Dh 1 83 $3 
we get 
2sj — 85 + Osg = As] (2.138) 
or 
a5 = 100287 = 4°3; (2.139) 


Setting the value of s{ in the above equation equal to unity, we get 
d” = 0.198 (2.140) 


And, from the previously given equations, 


iS = 1.802 
(2.141) 
s3 = 2.247 


The above characteristic vector can be written in the form where $3 = 
1.0 by dividing the components of the vector by sg = 2.247. This 
results in 


Sj = 0.445 S5 = 0.802 $5 = 1.0 


These values compare with previously obtained results for the lowest 
characteristic value. The normalized form of this characteristic vector 
can be obtained by dividing the above values by the square root of the 
sum of the squares. This gives 


sy. = 0.028. 55. = 0.501. 94 = 0.736 (2.142) 
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An additional method for the computation of the characteristic 
equations and the corresponding vectors will be discussed in the fol- 
lowing section. This method, referred to as the method of A. M. 
Danilevsky, is based primarily on the properties of similarity transfor- 
mation discussed previously. 


THE METHOD OF A. M. DANILEVSKY FOR 
OBTAINING THE CHARACTERISTIC EQUATION 


The method of Danilevsky for obtaining the characteristic equation is 
based on the reduction of matrix A, whose characteristic equation is 
desired, 


S17 - Oy 3 Ain 
Po ee er (2.143) 
anl a2 2,3 aan 


1 0 0 0 
iS ee ee 0 (2.144) 
0 Ge. 1 0 
where the elements p,, po, pg, -.-, Pp, represent the coefficients of the 


characteristic equation of matrix A. 7 

It can be shown that the transformation of matrix A to matrix Pcan 
be accomplished by a series of similarity transformations. Thus, on the 
strength of the theory that the characteristic equations of similar matrices 
are identical, the characteristic equations of matrices A and P will be the 
same. The fact that the elements of matrix P represent the coefficients 
of the characteristic polynomial is readily seen by writing the characteristic 


@) 
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determinant of P as 


Ppa he Bg Psat ®, 


1 oe ae | eee oa 
PAj= 0 th 8 (2.145) 
0 0 (a 


and expanding it in terms of the elements of the first row 
ee (-D"(2” eo =H.) ap (2.146) 


where D(X) is the desired characteristic polynomial and n is the order of 
the square matrix A. 

We demonstrate the transformation from matrix A to matrix P for a 
4 x 4 matrix 


at = ae a Oe (2.147) 


Postmultiplying Eq. (2.147) by the matrix M, which is formed from the 
elements of the fourth row of the matrix A, shown below 


1 0 0 9) 
0 1 0 0 
ey a ee 1 T44 (2.148) 
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will result in 


AM = | (2.149) 


where the elements b of the above are 


413% 41 413%49 ; Q13 
by, = 44, - s » Dy = Ayo - e ei ree 
43 43 43 
49394) @93%49 
bo, = 1 - » b59 = Ago - ’ 
43 743 


(2.150) 
Note that the last row of the matrix of Eq. (2.149) is the same as that of 
the P matrix of Eq. (2.144). The matrix AM of Eq. (2.149) is not as yet 
similar to matrix A of Eq. (2.147). It can be made similar to A by 
premultiplying by the inverse of matrix M. Thus, the similarity transfor- 
mation of A into the form of Eq. (2.149) is written as 


C = M-!AM (2.151) 


where M“! is readily obtained from Eq. (2.148) as 


0 1 0 0 (QH52) 


The third row of this matrix is the same as the last row of the matrix 


Eigenvalues, Eigenvectors, and Quadratic Forms 91 


of Eq. (2.147). The premultiplication of matrix AM of Eq. (2.149) by 


M!, Eq. (2.151), does not affect the last row of the matrix of Eq. 
(2.149). 


At this point the matrix C of Eq. (2.151) will be of the form 


(2. F353) 


eqs yo - “pg Eye 
Os; Gao @a5- “gap tei “go “ag .o4 
0 0 1 0 Gay Ff go: Cas faa 


(2.154) 
1 0 0 0 


go: Soa) ag Sao |} 
0 0 1 0 0 1 o) () 


Carrying the above process a step further will reduce the 4 x 4 matrix 
of Eq. (2.147) to the form of Eq. (2.144), that is, 


2a OS. 
a (2.155) 
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where p’s represents the coefficients of the characteristic polynomial 
of Eq. (2.146). 
As an example of the above procedure, consider the 4 x 4 matrix 


— 5.50988 1.87009 0.422908 0.008814 
0.287865 -11.8117 a. F119 0.058717 (2.156) 
0.049099 4.30803 -12.9707 0.229326 
0.006235 0.269851 1.39737 —17.5962 


whose characteristic equation is required. Following the above procedure 
the matrix M of the similarity transformation is calculated as 


1 0 0 0 
a 0 1 0 0 C2957) 
-—0.00462 -0.193113 0.71563 12.5924 
0 0 0 1 


The matrix M! is calculated as given by Eq. (2.152). That is, 


1 0 0 0 
| 0 1 0 0 (2.158) 
Met 
0.006235 0.269851 1.39737 -—17.5962 
0 0 0 1 


Having matrices M and M!, we implement the similarity transformation 
of Eq. (2.151) 
—~5.51177 1.78842 0.302646 5.33423 
0.262379 -12.9147 4.08761 71.9851 
0.185919 6.04616 -29.462 — 208.456 


0 0 1 0 
(22159) 


CAMS 
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At this point we set matrix C above identical to matrix A and repeat 
the similarity transformation about the third row of the above matrix. 
The M matrix for this case becomes 


1 0 0 0 
0.03075 0.165394 4.87284 34.4774 (2.160) 

: 0 0 1 0 

0 0 0 1 


The inverse of the M matrix can be obtained from the third row of the 
C matrix as 


1 0 0 0 
0.185919 6.04616 29.462 -—208.456 (2.161) 

4 0 0 1 0 

0 0 0 1 


The resulting matrix after the similarity transformation becomes 


— 5.56676 0.295794 9.01733 66.9944 
2.95252 -42.3217 — 562.559 — 2244.47 
0 1 0 0 


M-!CM = 


0 0 1 0 (2.162) 


We can set the above matrix identical to A and repeat the procedure. 
This, after obtaining matrices M, M~!, and M-!CM, will result in the 
matrix of the coefficients 


— 47.8885 -—797.281 -5349.48 -—12296.6 
1 0 0 0 (2.163) 
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From the above matrix and Eq. (2.146), the characteristic equation of 
the problem can be written 


D(A) = (~14(4 + 47.8885\2 + 797.281A2 
+ 5349.48A + 12296.6) = 0 


(2.164) 


The roots of this equation can be obtained by the methods described in 
Chap. 3. 


COMPUTER PROGRAM FOR THE METHOD OF 
A. M. DANILEVSKY 


The following computer program has been written in BASIC program- 
ming language (see [8]). This programming language includes matrix 
operations denoted by MAT at the start of each statement. The program 
starts with dimension statement 100 reserving space for matrices A, M, I 
(inverse of M), and P = AM. Statement 110 specified the order of the 
matrix whose characteristic equation is required. Note that in the 
BASIC language the first row and first columns of a matrix are denoted 
by row zero and column zero, respectively. For this reason N = 3 
specifies a fourth-order matrix. Statement 120 reads the elements of 
matrix A, from left to right, as given in data statements 350 through 
380. Statements 160 through 180 set matrices M, I, and P equal to 
identity matrix, zero matrix, and zero matrix, respectively. Statements 
190 through 220 form the similarity transformation matrix M. State- 
ment 250 obtains] = M7! matrix. Statement 280 obtains the P = AM 
matrix while statement 290 calculates M7! AM matrix and denotes this 
as the A matrix for the second iteration which starts from statement 
number 160 and so on. 

The results given for the example problem were obtained through the 
above described computer program. 

In exceptional cases the process of similarity transformation, as 
described above, may result in a matrix where a zero may appear in 
the position of an element whose value should be transformed to unity 
in the next iteration. Consider the C = M7! AM matrix at the end of 
some iteration to have the form 


100 
110 
120 
130 
140 
150 
160 
170 
180 
190 
200 
210 
220 
230 
240 
250 
260 
270 
280 
290 
300 
310 
320 
330 
340 
350 
360 
370 
380 
390 
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DIM A (10,10),M(10,10),1(10,10),P(10, 10) 
cet tie3 

MAT READ A\(N,N) 

PRINT “MATRIX A” 

MAT PRINTA 

Le b=) 

MAT M=IDN (N,N) 

MAT I=ZER (N,N) 

MAT P=ZER (N,N) 

FOR J=O TON 

LET M(L,J)= —A(L+1,J)/A(L+1,L) 

NEXT J 

LET M(L,L)=1/A(L+1,L) 

PRINT “MATRIX M” 

MAT PRINT M 

MAT 1=INV (M) 

PRINT “INVERSE OF MATRIX M” 

MAT PRINT | 

MAT P=A*M 

MAT A=I*P 

PRINT “---------- MATRIX A---------- a 
MAT PRINT A 

SD Ee Bie 

IF L<0O THEN 390 

GO TO 160 

DATA —5.50988, 1.87009, .422908, .008814 
DATA .287865, —11.8117, 5.7119, .058717 
DATA .049099, 4.30803, —12.9707, .229326 
DATA .006235, .269851, 1.39737, —17.5962 
END 


(2.165) 


with the element c, ,_, = 0. In this case a nonzero element is chosen 
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along the same row, c,,(i < k — 1)and the 7th and k — 1 columns and 
simultaneously the ith and k — 1 rows are interchanged. This operation 
will be equivalent to a similarity transformation with the M matrix 


i 
eo: 
OS = ith row 
| 1 ! 
M = ee (2.166) 
| | 
| cere clon k — lL row 
| 1 
ith Lae 
| 
col es 1 
col. 


The above matrix has the property M? = I, or M = M~!; therefore, post- 
multiplication and premultiplication of matrix C of Eq. (2.165) by M 
and M7! respectively, will represent a similarity transformation with no 
change to the characteristic equation. 


CHARACTERISTIC VECTOR BY THE METHOD 
OF A. M. DANILEVSKY 


Let » be one of the characteristic values of matrix A obtained from 
the characteristic polynomial derived above. The characteristic vector 


corresponding to A is denoted by X = (x, x9, x3, .-., x,) where small 
x’s are the components of the vector. The characteristic vector corre- 
sponding to the matrix P is denoted by Y = (Vis Vax 5 J, Ja Ee Oe 
shown that 

ee gs SY (2.167) 
where M,,Mo,...,M,_, are, respectively, the transformations which 


reduce the matrix A of Eq. (2.143) to matrix P of Eq. (2.144). The 
components of the characteristics vector of Eq. (2.144) can readily be 
obtained from 


\ 
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Py A; Po. Ps Pek Mt 
1 ast . 0 Yo 
0 —— . 0 |ly¥,|= 9 
0 0 = —X = 


(D1 = A)Y1 + Pate tt FY, = 0 


Wee jE = 0 


Setting y, = 1 we get ¥,-1 =>, ¥n-9 = 4%, 9, = A". Having these 
values and the product of M transformations, Eq. (2.167) will result in 
the characteristic vector corresponding to A. 

At this point a comparison of the previously discussed matrix iteration 
method and the method of A. M. Danilevsky is in order. In the method 
of Danilevsky, the characteristic equation is obtained and then the roots 
of this equation, the characteristic values, are calculated by numerical 
root finding methods of Chap. 3. Thus, the numerical proximity of the 
roots does not increase the number of required calculations as was the 
case in matrix iteration method. On the other hand, the matrix iteration 
method produces the largest eigenvalue and the corresponding eigen- 
vector at the same time. Since in many engineering problems the highest 
(or the lowest, which can be obtained by the inversion of the matrix of 
the coefficients) eigenvalue and the corresponding eigenvector of systems 
represented by real symmetrical matrices are required, the matrix iteration 
method under these conditions is preferred. 

The number of multiplications and divisions by the method of A. M. 
Danilevsky in obtaining the characteristic equation is (n - Dir +n =B 
where n is the order of the matrix. The method of A. N. Krylov for 
obtaining the characteristic polynomial requires a greater number of 
arithmetical operations than Danilevsky’s method. It is used primarily 
because of the simplicity of computational procedure. Another short- 
coming of Krylov’s method comes from the fact that arithmetical 
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operations are to be performed on numbers of different orders of 
magnitude, resulting, in many cases, in loss of accuracy. In cases where 
the characteristic equation has multiple roots, Krylov’s method results 
in the coefficients of lower-degree polynomial. 

The iteration methods of calculating eigenvectors and eigenvalues 
discussed above were primarily given from a computational point of 
view. Characteristic values and vectors, of course, have definite physical 
meanings in vibration problems as described in the following pages. 


CHARACTERISTIC VALUES AND VECTORS IN 
MECHANICAL VIBRATIONS — 


It will be shown in the following example that the characteristic 
values correspond to the natural frequencies of vibration in mechanical 
systems while the characteristic vectors correspond to the normal modes 
of vibration. If a given conservative (no friction) dynamical system is 
initially displaced proportional to one of its normal modes, the system 
will vibrate indefinitely with a frequency equal to the natural frequency 
of the system corresponding to that normal mode. 

Consider the dynamical system of Fig. 2.1. The moment of inertia of 
each disk is denoted by I while the torsional stiffness of the connecting 
shaft is denoted by c. The length of the shaft between disks is 7. The 
equations of motion of each disk from Newton’s second law of motion 
can be written 


-IG, = = (9 ~ Qo) + (Gp — Q3) (2.168) 


-IG., 


Il 


: (dg = Qo) 


poo ere 


figure 2-1 Vibrating shaft with three disks. 
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where q’s are the angular displacements and dots represent derivatives 
with respect to time. The above equations can also be written in the form 


-Iq, = (24 — Qo) 


es Cc 
-Iq5 rie + 2q5 = Q3) (2.169) 


-IqG, = (4 + Qs) 
Consider the solution of the above equations in the form 


Gq. = A,em lal aw) i = 1,23 (2.170) 


Obtaining the second derivative of the above equation and substituting 
in Eq. (2.169), we obtain 


Il -» 

BA egy : (H.2) 4, 

a eae Il 2\A 

hy «thy “tye, (2.171) 
if; 


The above homogeneous equation can have nonzero solutions for A,, Ao, 
and A, only if the following determinant vanishes 


~4 ZK Se 2 iis 


A = —©@ (2.173) 
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With this definition of } Eqs. (2.171) can be written in matrix notation 
as follows 


eee OT ; 
A 2 A (2.174) 
ee ee ae : 


The roots of the determinant of Eq. (2.172) were previously calculated 
to be 


A, = 3.288 Ay = 1.555 A, = 0.198 (24795) 


Using these values in Eq. (2.174) will result in corresponding values of 
A’s, which were also previously calculated as 


ope 
dh, = 3.238 i = -0.736 


Az = 0.328 
A, = —0.736 
hy = 1.555<A, = —0.328 (2.176) 
Ba = et 
i‘: = 0.328 
Ag = 0.198<A, = 0.591 


S. 
> 
wt) 
\| 


0.736 


Note that the angular frequency can be calculated from the values of 
d’s using Eq. (2.173); thus 
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oy = Lao y= 
L2atyf/= (742) 
Il 


Using the above values and the values A from Eq. (2.176) in Eq. (2.170), 
we obtain for the complete solution 


| oO 


sak 


qd, = 0.5914 sin (w, t+ ¥y) — 0.736B sin (wot + Bo) 
+ 0.328C sin (wet + Wo) 


dy = —0.736A sin (w,t + W,) — 0.328B sin (wot + Yo) (2.178) 
+0.591C sin (wt + Wg) 


dg = 0.328A sin (wt + w,) + 0.591B sin (wot + Wo) 
+ 0.736C sin (wgt + Wa) 


It should be remembered that the A’s of Eq. (2.176) represent the ratio 
of characteristic values rather than their absolute values. For this reason 
the constants A, B, and C are added to the above equations. 

In dynamics the values of w as given by Eq. (2.177) are referred to as 
the natural frequencies of vibration and corresponding values of A’s of 
Eq. (2.176) are modes of vibration. 

It can be shown that an initial displacement of the disks of Fig. 2-1 
proportional to the values of the modes of vibration (Eq. (2.176)) will 
result in oscillations of the system at the rate given by the corresponding 
frequency. | 

To prove the above statement, assume the following initial conditions 
for the system of equations (2.178): 


at t = 0 q, = 0.591 qo = A d3 = 0.328 


4, = 0 dy = 0 dad 
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Using the derivative condition above in Eq. (2.178) we obtain the 
following set of equations 


0.591 -0.736 0.828} | Aw, cos vy, 
-0.736 -0.328 0.591 | | Bos cos yy | = 0 (2.180) 
0.328 0.591 0.736 | | Cus cos Wy 


Since the determinant of the coefficients of the above equation is not 
zero, the only way that the equation will be satisfied is 


cosy, =0 , = 90° and ¢, = 9° Y= 90° (2.181) 


Using the initial condition for q,, dg, and gg from Eqs. (2.179) and 
(2.178), we get 


0.591 -0.736 0.328] | A 0.591 
—0.736 -0.328 0.591] ]| B | = | -0.736 (2.182) 
0328 O0891. 0.79611 C 0.328 
From the above equation 
foe to hae Ue a B (2.183) 


And, the system of equations (2.178) becomes 


q, = 0.591 cos wy,t 


dg = —90.736 cos w,t (2.184) 


q3 = 0).328 COS Wb 


Thus, the system will vibrate with the frequency @, once displaced by 
the normal modes corresponding to this frequency. 
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THE RELATION BETWEEN THE EIGENVECTORS 
AND EIGENVALUES OF A MATRIX AND THOSE 
OF ITS TRANSPOSE 


In this section we will consider the relation of the eigenvalues and 
eigenvectors of a matrix A and those of its transpose A‘. 
Let the eigenvector equation of the matrix A be 


AX = AX : (2.185) 
Let the eigenvector equation of A’ be 
AY = pY (2.186) 
The characteristic equation of the matrix A is 
P(A) = det (A - AD = 0 (2.187) 
The characteristic equation of A’ is 
Q(A) = det (A - pl) = 0 (2.188) 


However, since interchanging the rows and the columns of a determt- 
nant does not affect its value, we have 


P(A) = det (A — Al) = det (A — Al)’ = det (A —- AD (2.189) 
we see that the form of the characteristic equation of A is the same as 
the characteristic equation of A’ and therefore that A and A’ have the 


same eigenvalues. 
If we premultiply Eq. (2.185) by Y’, we obtain 


Y'AX = AYX (2.190) 


If we now take the transpose of Eq. (2.186) and postmultiply the result 
by X, we obtain 


Ax = pV Xx (2.191) 
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We now subtract Eq. (2.191) from Eq. (2.190); the result is 

(A- pYX = 0 Citas 
If the eigenvalues A and p are such that \ # p, we have the result 

YX = 0 (2493) 


It is evident from Eq. (2.193) that the eigenvectors of A are orthogonal 
to the eigenvectors of A’ belonging to different eigenvalues. This fact 
may be utilized in the determination of the eigenvectors of nonsym- 
metrical matrices that belong to repeated eigenvalues. 


THE GRAM-SCHMIDT ORTHOGONALIZATION 
PROCESS 


In certain applications such as the determination of the eigenvectors 
belonging to repeated eigenvalues, we are concerned with the problem 
of obtaining an orthogonal set of vectors from any given set of n linearly 
independent vectors. The procedure by means of which this object is 
achieved is known in the mathematical literature as the Gram-Schmidt 
orthogonalization process. 

In this process we are given a linearly independent set of vectors 
X1, X9,Xg3,---,X,. The problem is to find scalars Ci such that the 
vectors given by the scheme 


fee 

Yo = C9,Xq + Xo 

Y3 = €31X1 + CgoXo + Xz sie ay 
Y= City + CnoXo + = ee = 


are orthogonal. 
Let us introduce the notation 


(Y,,,X,) = Y,X; =< Y,: x; (2.195) 
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so that (Y,, x) denotes the scalar product of the vectors Y, and X ; Now 
consider the vectors 


y= ey 
Xo) 
a3 = Ap 4 = 
(Y,,Y,) 
oe Cpe 
ey l oe 27°38 (2.196) 
(Yee (Y5, Yo) 
eae Bee eee Se, 
Ly eke fap Mia ee ee as 
ye) ie, 
We note that 
(X,,X4)(X1, Xo) 
(Yon; Yo) = (X,, Xo) Bp eps SS ee ae 0 (2.197) 
(X,,X,) 
We can show similarly that (Y,, Y,) = 0 and that in general 
(Y ,, Y= © ae PP (2.198) 
Therefore the vectors Y,, Yo,---, Y, form an orthogonal set of vectors. 


Example. Consider the set of vectors 


X= 110,10) 
5 gee ee) 
gras Se aimee 


The orthogonal set of vectors Y are obtained using Eqs. (2.196) 
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Yo 8 010) 
eo = 41, L420 - (1, 1, 3,0), (1,9, 1,0 (1 9 1.9) 
(1,0, 1,0), CS Le 
= 4 455, 0) —- 201,60, 1.0) 
= (-1,1,1,0) 
267 MBB AAO ee is 


(-1,1, 1,0), (-1, 1,1, 0) 


— (0, 2,0, 0), (1,0, 1,0) 
(1,0, 1,0), (1,0, 1, 0) 


(1,0, 1,0) 


THE MODAL MATRIX AND THE 
SPECTRAL MATRIX 


Let the nth-order matrix A have n distinct eigenvalues 4,, 
f=A,2,...,n and n eigenvectors X,78.2:1,2,3....,0 Let tag 
eigenvectors X, be arranged as the columns of a square matrix M. This 
square matrix is called the modal matrix of A; it has the form 


n 


The modal matrix M is therefore a partitioned matrix whose columns are 
the eigenvectors of the matrix A. 

The spectral matrix of the matrix A is a diagonal matrix S of the nth 
order whose elements are the eigenvalues of the matrix A (we are 
considering here the case in which the matrix A has n distinct eigen- 
values). The spectral matrix S has the form 


eG 0 
eee ee 0 
$0 024, =... Mle tee. ee 


oeeee3ec3s3es3eestseeteeeeeeee eee 6 
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The typical eigenvector and eigenvalue X, and A, of the square matrix 
A satisfies the eigenvector equation 


ae Nae Se eee (2.201) 


Let the matrix A be postmultiplied by its modal matrix M to obtain the 
result 


Me A, Xe eR = 1A, Ay a A 
(2.202) 


pee ee ns 


The matrix M has an inverse provided that the eigenvectors of the 
matrix A, X,,k = 1, 2,3,...,” are linearly independent. This will be 
the case if A has n distinct eigenvalues \, or if A is a symmetrical matrix 
so that A’- A. If the matrix A is not symmetrical and some of its 
eigenvalues are multiple eigenvalues, the matrix M may not have an 
inverse. The result (2.202) AM = MS can be premultiplied by M in 
the case M has an inverse. We thus obtain 


M-'AM = S (2.203) 


Equation (2.203) is said to represent the diagonalization of the matrix 
A to a diagonal matrix. The equation AM = MS can also be postmultiplied 
by the inverse of the modal matrix M7! to obtain 


A = MSm7! (2.204) 


It is interesting to note that if Eq. (2.204) is multiplied by itself, the 
result is 


A2 — (MSM-!)(mMsM7}) = MS(M-!M)Ssm-! = MS2M7! (2.205) 


The matrix S? is a diagonal matrix whose elements are the squares of 
the eigenvalues of A. If we multiply Eq. (2.204) by itself k times we 
obtain 


At aS eS 1, 8. (2.206) 


The matrix S* is a diagonal matrix whose elements are the kth powers of 
the eigenvalues of A. 
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THE DIAGONAL FORM OF A SYMMETRICAL 
MATRIX WITH DISTINCT EIGENVALUES 


Let us consider A to be a symmetrical matrix with distinct eigenvalues. 
We now multiply the equation AM = MS on the left by the transpose of 
M, M’, to obtain 


MAM = MMS (2.207) 


We now take the transpose of Eq. (2.207), realizing that A’ = A, and 
obtain 


MAM = SMM (2.208) 
From Eqs. (2.207) and (2.208) it follows that 
MMS = SMM (2.209) 


Since S is a diagonal matrix, in order to satisfy Eq. (2.209) we must have 
M’M = a diagonal matrix. If the eigenvectors X , are normalized to unit 
length so that 


Lo er C2) 


ee: 


the modal matrix of the symmetric matrix A has the property that 
MM = 1 (2.211) 


where I is the nth-order unit matrix. Thus M is orthogonal in this case 
and the Eq. (2.204) takes the form 


A = MSM (2242) 


In Eq. (2.212) M is the modal matrix of A and M’ is its transpose. 
This equation is the diagonal form of a symmetric matrix with distinct 
eigenvalues. 

Diagonalization of a matrix is closely connected with the problem of 
obtaining quadratic forms. In many engineering applications, such as 
finding of the principal axis of inertia and coordinate transformations, it 


Eigenvalues, Eigenvectors, and Quadratic Forms 109 


is desirable to reduce a given expression with cross-product terms to a pure 
quadratic form, that is, to an expression involving squares of the variables 
without cross-product terms. It will be shown through the following 
example that this problem is similar to finding the appropriate character- 
istic values and vectors of the expression with cross-product terms as 
described above. 

Example. Obtain the pure quadratic form for the expression 


o = ea — 2x1 %_ + Gag — 2X5 Xg + ee (2.213) 


The above relation can be put in symmetrical matrix form as follows: 


2x4 X%q  —Xy Xo 0 
0 —XgXo X9Xg 


In the above, it is assumed that X ix; = XX; . Thus the resulting [A] matrix 
of the coefficients becomes 


y lees Bete |, 
A =f-1..2 -1 (2.215) 
1 Sees Ne 


Q = (x4 Xo9 X9) = | eres | Xo (2,216) 


Multiplying the last two matrices, we get 
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2x) — Xo 


which upon multiplication results in 


Q = X41 (2x4 a Xo) =8 Xo (—x4 a 2X5 se X) 35 X3(-Xo == X3) 
9 ‘ . (2.218) 
= 2X4 = 2x4 Xo + 2X5 = 2X9 Xe + Xe 


Equation (2.218) is identical to the expression for Q given by Eq. 
(2.213). This example was purposely chosen to yield matrix [A] whose 
characteristic values and vectors were previously calculated and are 
given in the normalized form as follows: 


A, = 3.238; [0.591, —0.736, 0.328] first column matrix [M] 
Ay = 1.555;  [-0.736, —0.328, 0.591] second column matrix [M] 
Az = 0.198; [0.328, 0.591, 0.736] ‘third column matrix [M] 


Using these values, the corresponding [M] matrix of Eq. (2.199) becomes 


0.591 -0.736 0.328 
M =| -0.736 -0.328 0.591 (2.279) 
0.328 0.591 0.736 


And, since [M] is an orthogonal matrix [M’ = M~!] the corresponding 
relation to Eq. (2.203) becomes 
0.591 -0.736 0.328 es 0 
=U.756 ~~ —~0.398 — 59) £ Po) 2-2 


Uae <8.591 0.736 a ee 


(2.220) 
0.591 -0.736 0.328 = 0 


—0.736 -0.328 0.591] =] 0 1.555 0 
G-a28- 0.591 - —0-736-4 0 0 0.198 
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Consider the following transformation from x to € coordinate system 


ae (soi —0.726: - 4284} ¢, 
%o 1 = |-0.736 -0.328 0.591] | é, (2.221) 
<i 0.328 0.591 0.736] | &, 


In matrix notation this becomes 

(x) = (M]() OP G2 2 
Transposing this, we get 

(x)’ = [E\ (M)’ (reversed order) (2.223) 


We now write Eq. (2.223) in full matrix form: 


0.591 -0.736 0.328 
(X41, Xp Xg) = (€4, €g, €3) |-0.736 -0.328 0.591 (2.224) 
0.328 0.591 0.736 


Using the expressions given for matrices X and X’ in Eqs. (2.221) 
and (2.224) in Eq. (2.216), we get: 


0.591 -0.736 0.328 ee ee | 
Q = (€,, €5, &3) |-0.7386 -0.328 0.591 See ee 


0.328 0.591 0.736 0 -l Q 


2.225 
0.591 -0.736 0.328] |é, er 


0.736 -0.328 0.591 | |, 
0.328 0.591 0.736 | | é5 
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Substituting the diagonal matrix of Eq. (2.220) in place of the three 
matrices at the center of the above expression, we get 


Q = 3.238¢,? + 1.555¢,2 + 0.1985? 


which represents the quadratic form of Eq. (2.213) with the transfor- 
mations of coordinates x to & as given by Eq. (2.221). The transfor- 
mation from the coordinate system € to x can easily be obtained by 
premultiplying Eq. (2.222) by [M]~1(= [M]’); this gives 


E€ = Mx (2.226) 


PROBLEMS 


1. Find the eigenvalues and eigenvectors of the matrices 


+422 : inv= = 
a CSA a 
y ua: 


2. Given the matrix 


—_— CO 


show that its eigenvalues are 
oe > See Page 
t= hy 2-2 8 


3. Given the matrix 


= show that A, = i. Ay = ie As = 2 


4. Given the matrix 


i. gee show that A, = 3, Ag = 3, and Ag = 12 


10. 


11. 


PZ. 


13. 


14. 


ES: 


16. 


Eigenvalues, Eigenvectors, and Quadratic Forms 113 


Find the characteristic polynomial of Prob. 2 by the method of 
A. N. Krylov. 

Find the characteristic polynomial of Prob. 3 by the method of 
A. N. Krylov. 

Find the characteristic polynomial of Prob. 4 by the method of 
A. N. Krylov. 

Find the highest characteristic value and the corresponding vector 
for the matrix 


: = Sees 

Pg 5 Answer: characteristic value 5.042 vector (5 
iterations) 0.555, 1.246, 1,000 

1 2 Z 


by matrix iteration. 

Find the smallest characteristic value of the above problem by 
iteration. 

Using orthogonality relations find all of the characteristic values 
and vectors of Prob. 2. 

Obtain the characteristic values and vectors of the matrix 


| ee ee 
ee ae 
a =2. 2 


by iteration. 

Obtain the characteristic equation and characteristic vectors of 
Prob. 2 by the method of A. M. Danilevsky. 

Obtain the characteristic equation and characteristic vectors of 
Prob. 3 by the method of A. M. Danilevsky. 

Obtain the characteristic polynomial and characteristic vectors of 
Prob. 4 by the method of A. M. Danilevsky. 

Obtain the characteristic equation of Prob. 8 by the method of 
A. M. Danilevsky. 

Solve the cubic equation obtained above and use the method of 
Danilevsky to obtain the characteristic vectors. 
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17. 


18. 


19. 


20. 


2. 
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Given the matrix 


cos (9) _ sin (@) 
. 
—sin (9) — cos (6) 
show that A is orthogonal, A°A = I, and determine the inverse of A 


and its eigenvalues and eigenvectors. 
Given the three linearly independent vectors, 


by the use of the Gram-Schmidt process construct a set of three 
orthogonal vectors Y,, k = 1, 2, 3 from ) Cie Cree. Ce 
Reduce the matrix 


78-66-55 
Aj = 9 = 
200 -160 43 


to the diagonal from M-! AM = S. 
Given the matrix 


ee Se 
A = oe ee 
aS es ee 


show that the eigenvalues are 2, 4, 6. Determine the eigenvectors 
of A. | 
Show that the eigenvalues of 


be 8 


23: 
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a a2 0) 
= ry +4) 0 e 
2 
0 re + 1) 5 
2 


are a, (1 + r)a, —ra. Find the corresponding eigenvectors. 
Show that the matrix 


ee ee ee 
es 2-8-8 
A = 
es ae | Se 
§ § =1--4 


is orthogonal. Find its eigenvalues and verify that its eigenvectors 
are “vectors of zero length.” 
Obtain the quadratic form of the equation 


Q = 2x,2 + 2x? — 15x,” + 8x, x9 - xox, - Lx, x3 


and the corresponding transformations. 
Hint: The characteristic values of the resulting matrix are —2, —18, 


and 9. 
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3 ROOTS OF 
POLYNOMIAL AND 
ALGEBRAIC EQUATIONS, 
LAGRANGE’S 
INTERPOLATION 
FORMULA, METHOD OF 
LEAST SQUARES 


The formulation of many engineering problems results in linear con- 
stant coefficient differential equations, the solutions of which can be 
obtained by writing the characteristic equation of the differential equation 
and solving for the roots of the resulting polynomial equation. In 
addition, the natural frequency of vibration in many electrical and 
mechanical systems is obtained by solving the characteristic equation of 
the system, which usually is an algebraic polynomial. In control systems, 
the stability of the system is determined by location of roots of the 
denominator of the transfer function of the system. In all of these 
cases, one is faced with solving for the roots of a polynomial equation. 
In a number of applications, solutions will result only by finding the 
roots of algebraic equations which, in addition to powers of the inde- 
pendent variable x, may include trigonometric and/or exponential 
functions (transcendental equations). 
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The inverse of finding the roots of polynomial equations, namely, given 
a number of data points, find a polynomial which passes through those 
data points, also occurs. In this respect, Lagrange’s interpolation formula 
is the one most often used. However, in problems where the data may 
include random measurement errors, the method of least squares is often 
employed. 


ROOTS OF POLYNOMIAL EQUATIONS 


Consider the nth-order polynomial equation P_, (x) of the independent 
variable x 


Pee PA) = Aes eG soe +++ + Ayx + Ao = 0 
(3.1) 


where the A’s are the coefficients of powers of the independent variable 
= 

Several numerical methods are available for determining the roots of 
this polynomial equation. However, before describing these methods a 
number of algebraic theorems dealing with the roots of polynomials are 
given for reference. The proofs of these theorems can be obtained in 
standard texts on algebra. 

Theorem 3-1. If (x — r) is a factor of f(x), then r is a root of the 
equation f(x) = 0. 

Theorem 3-2. If the polynomial f(x) is divided by (x — r), the re- 
mainder will be f(r). For the proof, let f(x) be divided by (x - r) and 
call the resulting polynomial Q(x) and the remainder R. This gives 


f(x) = (x —-nQ(x) +R (3.2) 
forx =r 
f(r) = R 
Theorem 3-3. If r is a root of the equation f(x) = 0, then (x — r) isa 
factor of the polynomial f(x). 


Theorem 3-4. A polynomial of degree n cannot have more than n 
roots. 
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Theorem 3-5. If f(x) is a polynomial and a and 6 are real numbers such 
that f(a) > Oand f(b) < 0, then f(x) has a root between a and b. 

Theorem 3-6. The coefficients of the polynomial are related to the n 
roots 1}, To, 13, -++9 Tp, by 


n 
An-1 ) 
= — Pa tg 
a i=l 
A 
n-2 
x = ee ee 
i<j 
= (3.3) 
A 3 2 
= — Fitts = stg totay = F4%a'go 
A, i<j<k 
A n 
oe 5 28 pop ee ee AT it Send. 
rie 1923 4 ree 
x oe A 


Theorem 3-7. If the coefficients A, of P,,(x)are real anda + bj (where 
j2 = -1) isa root of polynomial P_ (x) = 0, thena - bjis also a root. 

Theorem 3-8 (Rule of Signs). The number of positive roots of 
P(x) = 0, where the coefficients are real, cannot exceed the variation in 
sien of the coefficients of the polynomial P,(x) = 0. The number 
of negative roots cannot exceed the number of variations in sign of the 
coefficients of P,(—x). Moreover, in each case, if the number of roots is 
less than the number of variations in sign, it must be smaller by some 
multiple of two. 

Theorem 3-9. To transform an equation of the nth degree into an 
equation each of whose roots is the negative of the corresponding root 
of the original equation, we replace the independent variable x by -x in 
the equation. 

Theorem 3-10. To transform an equation of degree n into an equation, 
each of whose roots is less by k than the roots of the original equation, 
we proceed as follows: Divide the polynomial by (x — k) and denote 
the remainder by F,; divide the quotient by (x - k), and denote the new 
remainder by R,; continue until n + 1 remainders are found. Then 
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es el be oth ee eet oar SG Fe 


is the required equation. This can be accomplished through synthetic 
division as described below. 
SYNTHETIC DIVISION 

Consider the polynomial 


f(x) = P(x) = A jet rs Hens GS eo a Ay Bor AA, =D 


: (3.4) 


The following relations can be written using the coefficients of this 
polynomial 
Py (x)= xP (x) + Ap< 4 


P(x) = xP + A, (3.5) 


fit) = xP, a) eA 


0 
Multiplying the first, second, third, . . . equations above by 
x7, xn-1 yn-2 | respectively, and adding all of the resulting relations 


will give Eq. (3.4). In this operation, the terms cross referenced by 
arrows in Eq. (3.5) will drop out. Thus the value of polynomial P_ (x) at 
specific value of x = x, can be obtained by evaluating P)(x,)) through 
P(X) in the above relations. This can be accomplished by synthetic 
division and is considered simpler than substituting x, in Eq. (3.4) and 
evaluating powers of x). The synthetic division will result in 


Fe Fas Ag Ay Ay [Xo 
Xo Po (XQ) Xo P, (i, eS eg Be: 9S) be FFG 


P (x9) P, (Xp) Po(xy) --- P ipa) P_ (x) 
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where values of Ps through P, are sums of respective columns, for 
example, P, = x) P, + A,» and values at the second row are obtained 
by multiplying P,, P,, Pg, ... by Xo. 

Example. Determine the value of f(3) for the function 


f(x) = 4x° — 30x? + 6x? — 9 
by synthetic division. This results in 
4 SO = <6 0 =9 / 3 


a 48 eG 
, oe 2 Bo ae ee = AS) 


DERIVATIVE OF POLYNOMIALS 
BY SYNTHETIC DIVISION 


Differentiating Eqs. (3.5) with respect to x, we have 


Pe = 0 
P, (x) - P(x) + aw) = Po (x) = A, 
P,(x) = EPs (x) + P, (x) (3.6) 


Pio or, 34 FG) 


where primes denote derivatives with respect to x. 
Further differentiation of the above relations will result in 


ee = 
P(x) =H 


Po) = 4, 2 2P Gy = 2A, 


Po) = oP. is OP 
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Or 


Where? = 1, 2, 3, << 


Further differentiation of the above relations will result in the values 
of third- and higher-order derivatives 


Pa 
3! 4! 


These derivatives can be obtained by extending the synthetic division 
previously described: 


A, Ay Ais As Ay Ay Ao 7% 
XyPy XQ Py Kohn 4 Meh gcs ee 
Po P, Po ga race Pee P(x) 
XyP,  — X_HPo toP sg - So? a ee 
PY Py Ps see oe P), (xp) 
= 4 P3 ¢ oe . s5 
fe ee 2 
P re e POS PA 
ee ty 2! 2! 2! 
- P3 Py n-1 
Say = Fa) o 3 
es iS Pe Px) 
a 3! s SI 


These results, together with Taylor’s series expansion of P(x) around h 
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2 3 
P(x + h) = P(h) + xP’(h) 4+ se + ar euane Sess (3.7) 


where / is an incremental quantity, can be used to obtain polynomials 
whose roots differ from the roots of a given polynomial by the constant 
factor h. This in turn forms the basis of Horner’s method of finding 
real roots of polynomial expressions to be described shortly. 

Example. Given the example 


P(x) = x? — 12x? + 47x - 60 = 0 


write an equation with roots diminished by 2. If the roots of the above 
equation are r,, ro, ..., the roots of 


P(x + h) = x? + Blh)x? + C(h)x + Dh) = 0 


where B(h), C(h), and D(h) are functions of h, will be (r, — h), (ro Oss 
since P(r, -h+h) = P(r) =0. Thus, we need only to obtain the 
coefficients of the Taylor’s series (3.7) by synthetic division. This 
gives 


1 -12 47 -60 {2 


2 -20 54 
i —-- 77 +6 = P@ 


2 -16 
-= 3: 1 
2 


ee ee os 
1 = P°(2)/3! 


Having this, the desired equation becomes 


P(x + 2) = x? — 6x2 + 1lx - 6 = 0 
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HORNER’S METHOD OF FINDING REAL 
ROOTS OF POLYNOMIALS 


This method involves finding the approximate location of the real 
root of a polynomial and successively obtaining polynomials whose 
roots are diminished by these approximate amounts. This method was 
primarily devised for paper and pencil calculations. For calculators and 
computers it is much longer than other methods that will be discussed 
in this chapter. The method is best illustrated by an example. It is 
desired to find the roots of the equation 


f(x) = x2 + 9x? — 20x — 90 = 0 (3.8) 


To obtain the approximate location of a root, set x = 0,1,2,... . This 
will result in f(3) = —42 < O and f(4) = 38 > 0. Thus, by Theorem 3-5, 
f(x) has a root somewhere in the region 3 < x < 4. The next step is to 
find a polynomial whose roots are diminished by 3. That is, 


P=. & =29- 00 73: 


o--< O60. = 48 
cee SG Sa 
3 45 
i= 1p. 
3 

Ea 


This results in the polynomial 
f(x) = x? + 18x? + 6lx - 42 = 0 (3.9) 


The roots of this equation are between 0 and 1. Neglecting higher-order 
terms, let us take x = 42/61 ~ 0.7 and evaluate f(0.7) by synthetic 
division: | 


+. -18 61 — 42 {9.7 


0.7. --1300..- 61,563 
BS ee SS 9.863 = f(0.7) 
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From this f(0.7) = 9.863 > 0, let us try x = 0.6; this results in f(0.6) = 
1.296, again greater than zero. Let us now try x = 0.5; this will give 
f(0.5) = —6.875, which is negative. Thus, the root of Eq. (3.9) is 
between 0.5 and 0.6. Again by synthetic division we write an equation 
with roots reduced by 0.5 from those of Eq. (3.9). This results in 


18 61 — 42 / 9.5 


0.5 9.25 35.125 
A 1855 > 70.25 =6.875 


0.5 ao. 
i eee 
0.5 
ee 


with the equation 
fo(x) = x° + 19.5x” + 79.75x - 6.875 = 0 


The roots of this equation are between 0 and 0.1. Repeating the above 
process will give the final value of the root of Eq. (3.8) to three signifi- 
cant figures as 3.584. 


NEWTON’S METHOD OF SOLVING POLYNOMIAL 
AND TRANSCENDENTAL EQUATIONS 


This method of calculating the roots of polynomial and transcendental 
equations is used in both desk-type calculators and pencil and paper 
calculations. It involves successive approximations to the roots. New- 
ton’s method can also be extended to determine the complex roots of a 
polynomial equation. However, in this case the approximations should 
start with complex values. In this respect the method suffers from very 
complicated numerical work in obtaining the complex roots of a poly- 
nomial equation. The following is a description of the method for 
obtaining the real roots of transcendental and polynomial equations. 

Suppose that A is a real root of the equation f(x) = 0, and that A, is an 
approximation to A. Also, let us define the difference by h 
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1 
(3.10) 


Taylor’s series expansion for f(x) results in 


2 
f(A) = f(A, +h) = (Ay) + hf (A) + TAY = (3.11) 


But since A is a root of f(x), then f(A) = 0; neglecting terms containing 
h? and higher, we have 


fA) Hif ay = fiA)+ A— Adfa) = 0 


From this 


f(A,) 


— e 
f'(A,) 


1 


which is the next approximation to A. Continuing, we arrive at a value 
for the root. 

Example. Find the root of the equation f(x) = 2 cos x — 3x = 0. For 
x = 0, f(0) > 0; and for x = 7/2, f(7/2) < 0. For the first approximation, 
take midway between 7/2 and zero, that is, A, = 7/4 ~ 0.70. This gives 


f(A,) : 
VP _ 97 _ 2.¢08(0.7) - 2.10 


Bie eee = (0.567 
f (A,) 2 gind 3 


1 


The next approximation gives 


Further approximations can be obtained by repeating this process. 
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GRAFFE’S METHOD OF OBTAINING 
ROOTS OF POLYNOMIAL 
EQUATIONS 


This method is based on the fact that if the roots of a polynomial 
equation are different from each other, this difference will be exaggerated 
if the roots are squared. A polynomial each of whose roots is the square 
of the roots of the original polynomial can be written. Thus, by re- 
peating this process the roots of the resulting polynomials will be 
widely separated or, mathematically,r, > rg > rz _- “Where Ty, Yos Tay 4 
are the roots of the succeeding polynomials. As we will see, this makes 
it possible to use approximations to Eq. (3.3) as a basis of finding the 
roots. The Graffe method has the advantage of yielding the real as well 
as the complex roots of a polynomial. In addition, it results in all of the 
roots rather than one root at a time as in the case of previously given 
methods. As mentioned before the method is directly applicable when 
the roots are sufficiently different from each other. If the roots are 
close to each other in magnitude (or complex conjugate), special pro- 
cedures should be used to obtain their values. These procedures will be 
outlined later. 


Assuming the variation r; > ry > rz > r, --- in the roots of the 

polynomial 
ik) = 24 = laser ais ~ ee oes +++-+A,_,x+Ap = 0 
(3423 


Taking the dominant parts of Eq. (3.3), we can relate the above coeffi- 
cients and the roots 


A 1 A, -2 A, 3 


Lee 22 Pee 
n n n 


ves (3213) 


These relations are obtained from Eq. (3.3) by neglecting smaller-order 
terms. As mentioned before, the roots of any polynomial equation can 
be separated by squaring. Take as a preliminary example the quadratic 
equation 


x7 ~ 38x +2 = 0 (3.14) 
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with roots r, = 1, ro = 2. Squaring this equation will result in 


(x? 4 2)? = (3x)? 


2 ee Fe 8 
Substituting y = x* in this equation we obtain 


y" — By + 4 = 0 


with roots 1 and 4. Repeated squaring of the equation will give roots of 
1 and 16, 1 and 256, and so on. The corresponding values of the roots 
of the original equation can be found by using relations (3.13). This 
results in 


tae 2-20 yo 28 ro!) = 0667 


ye by + 4 =°0 FAS ye See r‘2) = 0.894 


PTs 1Gs= 0 .. 678. A = Bile el 


= 0.985 


(oO -2 Or (2 
257 


= 0.995 
(3.15) 
Thus, the resulting roots obtained by computing the coefficients are close 
to the roots of the original Eq. (3.14). 
The coefficients of Eq. (3.15) can be evaluated systematically as 
follows. Consider the polynomial equation 


II 
i) 


w? — 257w + 256 


X= Xe ae + bgx* = a foe ee + i = 


(Note that in this equation the order of subscripts of b’s is reversed as 
compared to Eq. (3.1). For example, the coefficient of the x”~! term in 
Eq. (3.1) was A, While in the above equation it is b,. This is done to 
ease the explanation which follows.) Transposing odd and even terms in 
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x to the right and left sides of the equal sign, we get 


‘os + a + pa + --) 


= = (0) 2"-! + ba + oe + +) 


Squaring both sides of this equation, we obtain 


ee eas gee oy ae alates (eae ee eee 
2n-6 2 
- x26 (b,? — 2by by + 2b, by - 2bg) +--+ = 0 
Substituting y = —x* in the above results in 
oo = yt 2(b,2 — 2b, by + 2b,) + --- = 0 


These coefficients can be obtained from the tabulated form 


First row 5 eee | b, bo bs b, 
Second row 1 b,? as Be ES 
Third row - 2b, —2b, b. —2bo by, — 2b. bs 
Fourth row 2b, 2b, bs 2bo be 
: -2b¢ —2b, by 
2b, 


2 2 2 2 2 
ey Ea 8 ee hb a 


+2b, +2b, br. +2bo be 
—2b¢ —2b, b, 
2b. 


The coefficients of x in the original polynomial are written on the 
first row. The squares of these are written on the second row. The 
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elements of the third row are obtained by multiplying the elements of 
the first row on either side and multiplying the result by —2. The ele- 
ments of the fourth row are obtained by multiplying the second element 
of the first row on either side and multiplying the result by 2. Similarly, 
the elements of the fifth, sixth, and other rows are obtained. The 
coefficients of the equation whose roots are the square of the roots of 
the original equation are obtained by summing the columns of the table 
from second row and below. Using this method for finding the coeffi- 
cient, the root-squaring method of Graffe can be applied in finding real 
and complex roots of polynomial equations, as illustrated in the following 
examples. | 
Example. Find the real root of the equation 


x3 _ 35x* — 206x + 240 = 0 (3.17) 


The table of coefficients for this problem can be written 


Root Coeff x? 
r 1 


i 
N 
. 
bd 
ee 
ja) 


=35 —206 240 


1.225 x 10° 4.24 x 104 5.76 x 10° 
0.412 x 10° 1.68 x 104 0 


r2 1 1.637 x 10° 5.92 x 104 S46 Scie" 


2.68 x 10° 3.50 x10? 3.32 x10? 
~0.12 x 10° =0.19 =< 10° 0 
r4 1 2.56 x 10° a ie 3.32 x 10° 


6.55 x 10!2 1.096 x 1029 1.102 x 1019 
~0.006 x 1012 ~0.0008 x 10!? 


6.54 x 10!2 1.095 x 10!? 1.102 x 1019 


From the last row we have 


H 


rel = VOS4 x 10!" = 30.09 ~ 


,/ 1-095 x 107° 
6.54 x 10/2 


oa) = 5.997 ~ 6 


11-102 x 10*” 
1.095 x 1019 


= 1.0009 ~ l 


> 
Co 
| 
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The root-squaring method is terminated when the resulting roots are 
sufficiently separated, that is, when the original condition of the method 
is fulfilled. Another way of stating this is whether additional squarings 
will separate the roots further. As seen from the above table the amount 
of improvement in the value of the roots is indicated by the cross- 
product terms, that is, —0.006 x 10!2 compared to 6.55 x 10!2 and 
~0.0008 x 10!? compared to 1.096 x 10!? in the last iteration. The 
squarings are terminated when the contributions of cross-product terms 
are considered negligible as compared to those of regular terms. 

The above absolute values can also be computed using logarithms. It 
can be seen by substitution in Eq. (3.17) that r, = 1.0009 is a root. 
From Eq. (3.3) we have 


Since r, is +1 from the last equation above, r, and ry must have different 
signs; and from the first equation r; should be +40 and rz should be —6. 


Complex Conjugate Roots 


For polynomials with real coefficients the complex roots occur in 
conjugate pairs. Since the magnitude of a pair of complex conjugate 
roots is the same, no amount of root squaring will separate these roots. 
In these cases, the real roots of the polynomial are calculated first and 
the complex conjugate pair of roots is determined by using the real 
roots and the products of complex conjugate roots. Assume that the 
following polynomial is obtained after several root squarings 


Baa Be “ee te 8, = 8 (3.18) 


Using Eqs. (3.13) the roots of the polynomial can be written 
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Eilat Bn -3 r Pn 3 
ae gee Bese : gee 
Be B-2 
Fate yc? Baa 4. r i 
ie Oo 4 7 r : ae 
B, Pn -2 (3.19) 


Assuming r, and r, to be real and ro and rz to be complex-conjugate 
pairs, the second of the above relations will involve imaginary numbers. 
In the root-squaring process this fact results in nonvanishing of cross 
products in the corresponding B, _ . term, as will be seen in the following 
example. In the Graffe method the presence of a complex-conjugate 
pair of roots is indicated by the nonvanishing of cross-product terms 
and many times also by frequent change of sign of these terms. 
Example with complex roots. Find the roots of the equation 


x3 + 94x? — 575x + 2,500 = 0 : (3.20) 


The table of coefficients for this equation becomes 


Root Coeff x? x2 x9 

r 1 94 —575 2,500 
1.15 x 10° — 4.700 x 10° 0 

6.25 x 10° 

igen 9.972 x 107 1.943x 10!° | 3.906 x 10!3 

0.028 x 107 —12.483 x 1010 

74 10.0 x 10’ 3.906 x 10!3 

From the last row 


Ir,| = ¥10® = 100 


Since this is the absolute value of the root, by substitution in Eq. (3.20) 
we note that r, should be negative r, = —100. The nonvanishing of the 
cross-product term in B, _ » (—12.483 x 10!°) indicates the presence of a 
complex root and since (r,ro)* = —10.54 x 10!9, then r. must be 
imaginary. By Theorem 3-7 r, must also be imaginary. Assuming that 
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[lp = Ut Jv 


ee A (3.21) 
Tg = U— JU 
we obtain 
ee ee 3.906 x 101% 
faba Ue 4 = = ¥/———___ =_ 20 
ry 108 
and since 
Pr, + tg +t, = 94 lo + To = 6 


This together with Eq. (3.21) will result in 2u = 6 or u = 3, and from 
u* + v2 =25,v=4. Finally, the roots of the equation can be written 


The presence of real roots of equal magnitude will similarly be indi- 
cated by nonvanishing of the cross-product terms. In general, the root- 
squaring method is unsatisfactory because of special rules it requires in 
case of multiple roots and complex roots. In the case of all real roots 
the application of the method is straightforward although the signs of 
the roots should be determined separately by direct substitution in the 
equation. In a given problem the real roots can be obtained by, for 
example, Newton’s method. Each root can be factored out of the 
equation by synthetic division and the process repeated on the poly- 
nomial of lower degree. If the polynomial contains only a pair of 
complex-conjugate roots, the quadratic obtained after extracting all of 
the real roots can be solved for complex roots. Another often-used 
method of obtaining the roots of polynomials by digital computers 
involves successive extraction of quadratic factors by iteration and the 
solution of the resulting quadratic equations for the roots which may 
be complex conjugate. The quadratic polynomials can be found by 
Lin-Bairstow’s method, described below. 
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LIN-BAIRSTOW’S METHOD OF 
QUADRATIC FACTORS 


Consider the polynomial 


ge 2 ee ee ee oe 
+ Agx* + A,x + Ay = 0 


It is desired to extract the quadratic factor x? + r*x + s* from this 
polynomial. If P_ (x) is divided by the trial polynomial x* + rx + s we 
obtain a quotient polynomial Q(x) of degree n—2 and aremainder Rx + S 

Px) = GA + 1% 18," + eee es + 5 9 es 
+ Bx = By) ae (3.23) 


FEquating the coefficients of the like powers of x in Eqs. (3.22) and 
(3.23), we get 


f= 
- ye 
Ae ce i. ig tosige 3h, (3.24) 


= R + rB, + sBy 


e 
| 


The coefficients B of the factored polynomial and the terms R and S 
are functions of r and s. Assuming that x? + r*x + s* is a factor of 
the polynomial Px, then 


Rift, s*) = 6 SO*,s% = 0 (3:25) 


To find r* and s* we take values of r and s near these values, so that 


r® = r+ Ar 


(3.26) 


s* = s+ As 


Polynomial and Algebraic Equations, Lagrange’s Formula, Method of Least Squares 135 


Using Taylor series expansion we have 


But so BG a) Aes Ase = 6 
or Os 
B27) 
S(r*,s*) ~ S (r,s) + ee - As 2 == 4) 
or Os 


where the derivatives are evaluated at r and s. Equation (3.27) gives two 
equations in two unknowns Ar and As. Use of these values in Eq. (3.26) 
will give the next approximation to r* and s* and the process is repeated 
from this point. 

The above procedure can be mechanized for a digital computer solu- 
tion as follows. From Eq. (3.22), if we use the last three terms as the 
first approximation of the quadratic form, we get the value of r and s 


Ay (3:28) 
A 


Equation (3.22) can also be divided by x”~ 2 and the resulting second- 
order equation used for the initial approximation of the quadratic 
form. This results in the values of r and s 


<< (3.29) 


Having the values of r and s, solve Eqs. (3.24) for B’s 
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3 (3.30) 
er 7 n-1 i, oa 
a = A; — rB, = sB, 
s = A, - rB, - sB, 
From the last of Eqs. (3.24) compute 
foes sar “aseB 
gee 265 
S = Ay = sBo 


Noting that B’s are functions of r and s, the partial derivatives of R and 
S with respect to these variables can be computed as follows: 


OR OBo OB, 
— = [85 +7 —J] -— s—— 


or or or 
= eo J ae — B. =. 
‘ : a (3.32) 
as as OB, 
Or Or 


OB 
OS (6, : =) 
Os Os 


The above equations involve the partial derivatives of B’s with respect 
to r and s. Denoting these partial derivatives by C’s, we have, from 
Eqs. (3.30)* 


*Note that A’s are not functions of r and s. 
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(3.33) 


gi =; OB. == 6B. =. 
ees aes (saree nae) ere 
Or or 


ea -(B, ogre Ts ea I cs gS 
Cy = -(B, + rC3) - sC, 


Partial derivatives of B’s with respect to s, D’s, can also be obtained in a 
similar fashion 


OB 
5 ew 

Os 

| eee OB 
ee # nm i ee a 0 

Os Os 

oo, sat te 5 eee (3.34) 
Di = Be a Bes costo ae = hoon eee 

S 
ae JP re aes sD, -i+2) 


OB, OB OB, 
Do ee a B, + s— = -1D, — (B, + sD.) 
Os Os Os 


Substituting these in Eqs. (3.27) and using Eq. (3.32), we get 


Ar [- (By + 1C,) - sC | + As [- (By + sD.) - rD» | 28 


(3.35) 
Ar [-sC, | + As [-(Bp é sD,)| = = 5 
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Denoting the coefficients of Ar and As by T, U, V, and W in the above 
equations, we get 


Ar{T) + Ast) = =R 
(3.36) 
Ar(V) + As(W) = -S 
Solving these simultaneous equations for Ar and As, we have 
i -RW + SU 
TW — UV 
(3.37) 
—e -TS + VR 
TW — UV 


These corrections are applied to the assumed values of r and s and the 
process is repeated from relations (3.30). The iterations are stopped when 
sufficient accuracy is achieved between successive values of r and/or s. 
Having the values of r* and s*, the roots are calculated using the 
relation 


—r# + \/(r*)(r*) — 45* (3.38) 
Roots = a 


Since with these values of r* and s* the remainder terms R and S of 
Fq. (3.23) are zero, the values of B’s will represent the coefficients of 
a (n — 2)-degree polynomial. The roots of this polynomial can be 
calculated by repeating the above procedure. 


TYPICAL COMPUTER LOGIC FOR THE 
MECHANIZATION OF LIN-BAIRSTOW 
METHOD 


The above equations can be written in flow chart from and subsequently 
programmed for digital computer solution. The following flow chart 
describes the logical evaluation of the given equations. The computer 
program is written in BASIC [4] programming language for a GE 235 
time-sharing computer. 

The following computer program mechanizes the logic given in the 
flow chart. The statement numbers corresponding to the blocks of the 
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Flow Chart of Lin-Bairstow Method 


Enter the degree of polynomial and 
the coefficient into the computer 


Compute initial values of 
rand s (Eq. (3.28)) 


ComputeB, C,and D 
(Eqs. (3.30), (3.33), (3.34)) 


Compute R, S, Ar, As 
(Eqs. (3.31), (3.37)) 


Correct values of rand s 


iS..7 = Toy 


Is Ar sufficiently small? 


Yes 
Compute roots 
: (Eq. (3.38)) 


Put the polynomial with B 
coefficients of Eq. (3.23) in 
the form of P_, (x) of Eq. (3.22) 


New polynomial 
degree 2 or less? 
Yes 
Solve the second- or first- 
degree polynomial 


Statements 
110 through 170 


Statements 
200 and 210 


Statements 
220 through 320 


Statements 
330 through 400 


Statements 
410 and 420 


Statement 430 


Statements 
450 through 490 


Statements 
500 through 550 


Statements 
560 through 570 


Statements 
580 through 610 
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flow chart are given next to each block. The roots of the following 
polynomial were calculated with this digital computer program 


x° + 3x4 — 4x3 _ 28x? 4 48x + 65 = 0 (3.39) 
The roots as given at the end of the program are 
Pgs 8S Zy 


and —1, where j? = —1. Some of the notations of the computer program 
are different from those used in the equations. These are 


Equation notation r S R S Ar As 
Computer notation R oe S| hil Re 2 


From the above discussion it can be concluded that the Lin-Bairstow 
method of computing the roots of a polynomial equation is adaptable 
to digital computer programming logic. It is also seen that this method 
gives complex roots of polynomials without having to deal with complex 
numbers during the course of calculation. 


COMPUTER PROGRAM FOR LIN-BAIRSTOW’S 
METHOD 


100 DIM A(10),B(10),C(10),D(10) 
110 READ N 

120 PRINT “DEGREE” N 

130 PRINT “COEFFICIENT” 
140 FOR!=NTO OSTEP —1 
150 READ A(1) 

160 PRINT A(1); 

170 NEXT I 

180 PRINT 

190 PRINT 

200 LET R=A(1)/A(2) 

210 LET S=A(0)/A(2) 

220 LET B(N)=A(N) 

230 LET C(N)=0 

240 LET D(N)=0 

250 LET B(N-1)=A(N-1)-R*B(N) 
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260 LET C(N-1)=—B(N) 

270 LET D(N-1)=0 

280 FOR I=2 TO N—2 

290 LET B(N-1I)=A(N-I)-R*B(N-I+1)-S*B(N-I+2) 
300 LET C(N-I)= -B(N-I+1)-R *C(N-I+1)-S* C(N-I+2) 
310 LET D(N-I)= -B(N-I+2)-S*D(N-I+2)-R * D(N-1+1) 
320 NEXT I 

330 LET R1=A(1)-R*B(2)-S*B(3) 

340 LET $1=A(0)-S*B(2) 

350 LET T=-B(2)-R*C(2)-S*C(3) 

360 LET U=-B(3)-S*D(3)-R*D(2) 

370 LET V=-S*C(2) 

380 LET W= -B(2)-S*D(2) 

390 LET R2=(-R1*W+S1*U)/(T*W-U*V) 

400 LET S2= (-T*S1+V*R1)/(T*W-U*V) 

410 LET S=S+S2 

420 LET R=R+R2 

430 IF ABS (R2) <.000001 THEN 450 

440 GOTO 220 | 

450 LET G=R*R-4*S 

460 IF G<0O THEN 490 

470 PRINT “ROOTS”-R/2"+OR-"SOR(G)/2 
480 GOTO 500 

490 PRINT “ROOTS”-R/2” +OR-"SOR(-G)/2’'J” 
500 LET N=N-2 

510 PRINT 

520 IF N=0 THEN 630 

530 FOR I=N TO 0 STEP—1 

540 LET A(I)=B(1+2) 

550 NEXT | 

560 IF N>2 THEN 200 

570 IF N<2 THEN 610 

580 LET R=A (N-1)/A(N) 

590 LET S=A (N-2)/A(N) 

600 GOTO 450 

610 PRINT “ROOT”-A (N-1)/A(N) 

620 DATA 5,1,3,-4,-28,43,65 


630 END 
DEGREE 5 
COEFFICIENT 

1 3 -4 -28 43 65 
ROOTS 2 +OR - 1. J 
ROOTS-3 +OR - 2. J 


ROOT —1. 
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LAGRANGE’S INTERPOLATION FORMULA 
AND INVERSE INTERPOLATION 


Assume that ¥5, 94, ¥9,--->¥, are values of y = P(x), an unknown 
polynomial, at x), x), %9,-.--,%x, Where the intervals between the x’s 
(x;, 1 — x,) are not necessarily equal. The problem is to find the poly- 
nomial P(x) which passes through (Xq, Vo» (X41 ¥4)s (Xa, Vo), -- = Fae 
Having this polynomial, the values of y for any other x can be obtained. 
This polynomial is called Lagrange’s interpolation formula. 

Let A,(x) be a polynomial which is zero for all x’s, except 4, 5 TOF 
which it is unity, that is, 


A, (x) = 


This polynomial can be written in factored form as follows: 


jo Re Se) 


sil n 


(3.40) 


A (x)= Acie = K(x = 4) - ++ ( — x, Ve — & 


Note that the (x - x.) term is missing on the right of this equation. 
Setting the value of A, (x ;) = | in the above, we can solve for A : 


1 


x“ Mpl(X — hy) «ss (x. = FS aie ee 


—— 
yaa 


Substituting this expression for A, in Eq. (3.40), we get 


as (7. — Xy(x - x4) --- (x - x, _ 4) (x - eg eee 
: GS Xo) (x. Sy eS 75> Keg p(y -— x) 


From this we can obtain Ay, Ay, Ao» Ag, .-. By-Setiing s =0, 1-a5- > 
This results in 
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fe = ee 
Ay (x) = ; : 
(Xp — X4) (XQ — XQ) see (x — x,) 
Cee a eS ee ee ee 
a es ee 
(65 = Xo) (x4 age Be te x) 
(3.41) 
(x tO a tyle (e = Hj = 2,44) > Be 
A, (x) cS 0.0.-— oo 
(g. — B(x, = ey) + Ge, — HG, - %, 2) Oe 
Tees ie ees eg ee See, 
A, (x) 0 1 n-l 
(x = Xe, = xy) OG = X,_ y? 


Note that in the ith equation (i= 1, 2, 3,...) the (x - x,) term is missing 
in the numerator. Now we form the polynomial P(x) using the coeffi- 
cients given in Eq. (3.41) 


n 
P(x) = A (xy, = Ap@)%p + Ay A, (x) ¥, 
s=0 


(3.42) 


Using the relation A, = 0 for s # k and A, = 1 for s = k, we get 


Pin = y Ag = A,(x,)¥, = Vp 
s=0 


which is the desired relation for the value of the polynomial at x,. 
Thus, P(x) passes through all points (x,, y,) where k = 0, 1, 2,.. . and 
the polynomial given by Eq. (3.42) is the desired Lagrange interpolation 
formula for unequal intervals in x. 

The coefficients given by Eq. (3.41) are considerably simplified for 
equal intervals of the variable x. To obtain these relations consider 
the following equations: 
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x = XQ + sh 


% = Xq + uh 


(3.43) 


where s is an integer, u can be a fraction, and h is the interval between 
two consecutive x’s. Using these relations the coefficients of the poly- 
nomial in Eq. (3.42) become 


fg) ge = 2) eee 


i, oe 
A, =D = 
Ay = (ht ee 

Ll ge-3 
_, 4 Du -2)>.-. u- GD 
Aa) ye: te De ee 
i'(n — i)! 
ude Dla 2) le = ae 
js meee (3.44) 


n! 


Note that in the above relations s and u refer to the position of x , and x 
with respect to x, as given by Eq. (3.43). 

Example. As an example of the application of Lagrange’s interpolation 
formulas, consider the evaluation sinx for x = 1.2 radians, given the 
following values 


Interval x 
h radians sin x 
XQ = 1.00 Yo = 0.84147 
a 
ee oie 


0.20 Crem Re 


ie 2130. _ 5, 2 Oa 
0.10 << 
So =I y, = BSS 
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Using the interpolation equations (3.41) for unequal intervals, we have 


(1.2 —1.IKE? -1.3)0..2 -— 14) 1 
(Ne eee 
Ge -400)- 1900-14 6 
(1.2 — 1.0)(1.2 - 1.3)(1.2 - 1.4) 2 
A, (x) = sees 
(ht 14d).t — b£2c.i.- 1.4 3 
(1.2 — 1.0)(1.2 -— 1.1)(1.2 - 1.4) 2 
Ag (x) eee 
(1.3 — 1.0)(1.3 - 1.01.3 - 14) 3 
‘= = a=. 1 
Ag(sy = 22= 100.2 = LUL2 = 1:3) __ 1 


(1.4 — 1.0)(1.4 - 1.1)(1.4 - 1.3) 6 
Using these values in Eq. (3.42) we get 


— Sin X| 4+ +o 7t.2 


= - = (0.84147) + = (0.89121) + = (0.96356) — _ (0.98545) 


= 0.93203 


The corresponding value of y given in the trigonometric table is y = 
0.93204. 


Inverse Interpolation 


The Lagrange interpolation formula can be used for inverse interpola- 
tion where the values of ¥), ¥1, Yo, ---» ¥, and the corresponding values 
Of Xp), X1, X9,---, Xx, are given and it is desired to find the value of x for 
a specified value of y. Since in most of these cases the values of y are 
not in equal intervals, Lagrange’s formula for unequal intervals is used. 
The procedure is the same as outlined previously. Note that the Gregory- 
Newton and Sterling formulas given in Chapter 5 can also be used for 
interpolation and inverse interpolation. 


METHOD OF LEAST SQUARES 


The method of least squares is used to find the closest (in a least- 
squares sense) polynomial approximation to a given set of data. Assume 
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that a set of data consisting of m + 1 points 


(Xqs Vos (Xz, Nz)s (XQ. Yo), «++» (XK, ¥,,) (3.45) 


is given. We assume a polynomial of degree n < m 
y = P(x) = Aw! 3 As hee Aa AL (3.46) 


and evaluate the coefficients A, through Aj by a least-square fit. For 
this we write the expression for the sum of the squares of the errors in y 
evaluated by using the assumed polynomial and the given data. That is, 


The values of A, through A, can be obtained by minimizing the value of 
error E with respect to the coefficients Nee See ee ne This is done 
by setting the partial derivatives of E with respect to these coefficients 
equal to zero 


) 


a 
aA 
n 
Ei sp 
OA, 4 (3.48) 
Ee zig 
dA, 


This will result in n + 1 equations in n + 1 unknowns A, through Ao. 
Solution of these equations will give the coefficients of the assumed 
polynomial P (x). 


Polynomial and Algebraic Equations, Lagrange’s Formula, Method of Least Squares 147 


y = (9/14)x + 2/7 


@Data points 


figure 3-1. Least squares fit to data points. 
Example. Consider data points (1,1), (3,2), (4,3) to be given in a 


cartesian coordinate system as in Fig. 3-1. It is desired to least-square-fit 
these points by a straight line 


y = Ax +B 


The expression for the sum square error becomes 


Error = E = [1 — (A + B)]2 + [2 — (3A + B)]? + [3 — (4A + B)I? 


The partial derivatives of E with respect to A and B will result 


a a oe 4 2 BS 
aA 
oe ee. eee 
JE 
ap 7 BL - A - B) - 202 - 84 - B) - 2(8 - 4A - B) = 0 


=6 + 8A + 3B. = 0 
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Or 


26A + 8B 
8A + 3B 


19 


i 
op) 


Solving the above equation for A and B, we obtain the coefficients of the 
required polynomial 


= ® 


Given the data points expressed by Eq. (3.45) and the form of the 
polynomial (3.46), a general expression can be derived for the coefficients 
of the set of simultaneous equations with unknown A - Ay: The 
simultaneous equations are obtained by writing the expressions for the 
derivatives of Eq. (3.48) and collecting terms. For example, the partial 
derivative dE/ 0A, will result 


+ 2(A, x1" : Ae fas at Ao = yy) x" = 
tae A se eee = > Ay = esa = 0 


Gathering terms we obtain the first equation in the set of simultaneous 


equations for A, - Aj, 


2n 2n 2n 
AK ee eee ] 
Ale = x 2n- 1 SS 4 


(3.50) 
fata Ay eat + My” FF i] 


ae n n 
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The rest of equations can be obtained in a similar manner by setting the 


derivatives dJE/dA,_, --- 0E/dAg of Eq. (3.48) equal to zero. The set of 
simultaneous equations for A,, — A, can then be written 


A [2n] + A, _,\2n = 


- + Ap [n] = [y(n] 
A [2n ete A, _,[2n See + 


+ Aylin - = Lyn —- DI) (3.51) 
A [n] 4. A. yln ox Ba et A, l0] = [y(0)] 


where the notation 


(3.52) 
and 


m 


d »:x/ 


1=0 


Ly (9)] 


is used. Denoting the matrix of the coefficients of A’s by the square 
matrix S and the right side of Eq. (3.51) by the column matrix T, we get 
SA = T 


(3.53) 
where A denotes the column matrix of the coefficients A, - Ao: This 


matrix can be obtained by premultiplying Eq. (3.53) by the inverse of 
the S matrix, that is, 


<2 Se 


The inverse matrix can be obtained, for example, by the augmented 
matrix method described in Chap. 1. 
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COMPUTER PROGRAM FOR LEAST- 
SQUARES METHOD 


The following computer program has been written in FORTRAN (see 
[5]) programming language to implement the calculation of the coeffi- 
cients in the least-squares polynomial fit. Statements 100 and 110 
reserve computer storage space for the following variables. 


Program Symbol Test Symbol Description 

X ¥ Data points 

y y Data points 

'&: Li] Eq. (3.52) 

B [Ly WI] Eq. (3.52) 

S S Matrix of the coefficients, 
Bq: £3:533 

T Ey Eq. (3.53) 

A A Matrix of the coefficients, 
A, Z, Ay 

CY The values of the variable 


y computed from the 
fitted polynomial 
N Degree of the polynomial 
M The number of data points 


Statement 120 reads the degree of the polynomial to be fitted N and 
the number of data points M. Statement 130 decreases the number of 
data points by one to compensate for starting from Sas Vg Sys Bee 
and so on. Statement 140 reads the coordinates of the data points to be 
fitted. Statements 150-250 compute the values of the coefficients given 
by Eq. (3.52), C’s and B’s of the program. Note that the symbol t 
specifies power. Statements 260-340 set these values in the matrix 
form of the S matrix of the coefficients. Note here that since the 
elements of the matrix start from the first row and the first column, the 
number N representing the degree of the polynomial is incremented by 
unity in statement 260. Statements 350 through 650 are exactly the 
same as the one given in Chap. | for augmented matrix inversion method. 
At statement 660 the inverse of the S matrix will be calculated and 
available. Referring to the augmented matrix method and subsequent 
program of Chap. 1, we see that the inverse matrix is stored in columns 
NX = M+ 1 and NY = 2N rather than 1 to N. This fact is used in state- 
ments 660 through 700 in calculating the product S~! T which represents 
the coefficients A, -—Ap- Statement 710 prints these coefficients. 
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100 DIMENSION X(25),Y(25),C(25),B(25),S(25,25) ,T(25),A(25) 
110 DIMENSION CY(25) 
120 READ, N,M 


130 M=M-1 

140 READ, (X(1),Y(1),1=0,M) 
150 NY=2*N 

160 DO 10 1=0,NY 

170 C(I)=0. 


180 DO 10 J=0,M 
190 C(I)=C(1)+X(J)T I 
200 10: 

210 DO 201I=0,N 
220 B(I)=0. 

230 DO 20 J=0,M 
240 B(I)=B(I)+Y¥(J)*(X(J) 1) 
250 20: 

260 N=N+1 

270 DO 30 J=1,N 
280 DO 301=1,N 
290 Nl=2*N-I-J 

300 S(I,J)=C(N1) 
310 30: 

320 DO 35 I=1,N 
330 T(I)=B(N-I) 

340 35: 

350 I=1 

360 NX=N+1 

370 NY=2*N 

380 DO 40 J=NX,NY 
390 S(I,J)=1. 

400 |l=i+1 

410 40: 

420 L=1 

430 K=2 

440 42: XM=S(L,L) 
450 DO 45 J=L,NY 
460 S(L,J)=S(L,J)/XM 


470 45: 
480 DO 50 I=K,N 
490 XZ=S(I,L) 


500 DO 50 J=L,NY 

510 S(I,J)=S(I,J)-S(L,J) *XZ 
520 50: 

530 L=L+1 

540 K=K+1 


152 


550 
560 
570 
580 
590 
600 
610 
620 
630 
640 
650 
660 
670 
680 
690 
700 
710 
720 
730 
740 
750 
760 
770 
780 
790 
800 
810 
820 
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IF (L-N) 42,42,52 
52: L=N 

53: LZ=L-1 

DO 60 K=1, LZ 

i=i-K 

YZ=S(I,L) 

DO 60 J=L,NY 
S(1,J)=S(1,J)-S(L,J)*¥Z 
60: 

2 ee 

|F(L-1)65,65,53 

65: DO 70 I=1,N 
A(1)=0. 

DO 70 J=NX,NY 
A(1)=S(1,J5) * T(J-N)+A(1) 
70: 

PRINT,(A(I),1=1,N) 
PRINT, tttttt 

DO 80 J=0,M 

CY (J)=0. 

DO 75 |=1,N 

CY(J)=CY (J)+A(1) *(X(J) T (N-1)) 
75: 

80: PRINT,X(J), ¥ (J), CY (J),CY (J)-¥ (J) 
$DATA 

2,3 

1,1,3,2,4,3 

END 


Statements 730 through 780 calculate the values of y at data points 


Xp — * 


> using the polynomial coefficients just developed. The results 


of the computer run for the case of the previously given example are 
shown below. 
Coefficients for a first-degree polynomial 


A, = 0.6429 A) = 0.2857 


Error 
ydata Ycomp Ycomp ~ Ydata 
1.00 0.9286 —0.0714 
2.00 2.2143 0.2143 


3.00 2.8571 —0.1429 
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The coefficients of the polynomial fit can equally well be computed 
for a second-degree polynomial. In this case we have 
Coefficients for a second-degree polynomial 


A; = ies? “A, = -0.1667- A, = 100 


and 
Error 
D Yaata Yeomp Yoeomp — Vata 
1.00 1.00 1.00 0.00 
3.00 2.00 2.00 0.00 
4.00 3.00 3.00 0.00 


This result is expected, since a second-degree polynomial will pass 
through the three data points. 


PROBLEMS 

1. Determine the number of positive and negative roots of the equation 
x2 — 3x44 4x3 4 3x2 4 6x + 3 = O 

2. Find the value of the function 
fia) = x? 4 Ge + 8x + 2 
at x = 3 and 5 by synthetic division. Check your solution by 
substitution in f(x). 

3. Determine the polynomial each of whose roots is diminished by 
2 from the roots of the polynomial 
x3 — 8x2 +4 19x + 12 = 0 

4. Find the root of the polynomial 


x? _ 5x2 + 6x — 1 = O 


between 1 and 2 by Horner’s method. Answer: 1.555. 
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5. Find the root of the equation 


by Newton’s method. 

6. Find the roots of the equation given in Prob. 3 by Graffe’s 
method. 

7. Find the roots of the equation 


i-j 
x3 _ 5x2 + 8x —- 6 = O Answer: | i-- j 
3 
by Graffe’s method. 
8. Find the roots of equation 
ee ee a oe Answers: 3.2469, 1.5549, 
.19806 
by Graffe’s method. 
9. Find the roots of the equation 
we 2 ie = 8 Answer: —2.3375, 3.7702, .5674 


by Lin-Bairstow’s method. 
10. Find the roots of the equation 


x3 z 94x2 = ik > 2,500 = 0 Answer: =i. 3H 4j 


by Lin-Bairstow’s method. 
11. Find the real roots of the equations of Probs. 9 and 10 by Newton’s 


method. 
12. Given the table of values 
n Vn 
150 12.24745 
152 12.32883_ | 4 
154 12.40967 
156 12.49000 


| 

! 
158 12.56981 : ‘ 

a 


160 P6411 


13. 


14. 


iS, 
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find V155 using Lagrange’s interpolation formula for equal intervals 
using the A and B sets of values, respectively. 
Using the table of values 


n vn 
50 3.684 
52 3.732 
54 3.179 
56 3.825 


find the value of n given V/n = 3.756 using Lagrange’s interpolation 
formula (inverse interpolation). Answer: 53. 
Given the table of values 


WNe Oo | 
freon 
= 


find the coefficients of the equation y = Ax? + Bx + C fora least- 
square fit to the given points. Answer: y = 3x2 + 2x +6. 

Find the coefficients of the equation y = Ax? + Bx + C for a 
least-square fit of the table of Prob. 13. Assume that x = n and 
y = vn. Answer: y = ~.000125 x? + .03675 x + 2.159 


16. Verify the terms of Eq. (3.51) for n = 2 and m = 3. 
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4 TIME-FREQUENCY 
DOMAIN ANALYSES AND 
THE FAST FOURIER 
TRANSFORM 


In engineering analyses, parameters of system behavior are either 
expressed as a function of time or as a function of the frequency 
contents. For example, the solution of differential equations of motion 
of rigid bodies will result in the values of displacements, velocities, and 
accelerations as a function of time, while oscillations of electrical and 
mechanical systems and doppler shifts of radar return are expressed in 
the frequency domain. Time and frequency domain analyses are related 
through Fourier series and transforms. Given a variable expressed as a 
function of time, Fourier analysis will decompose the variable into a 
sum of oscillatory functions, each having a specific frequency. These 
frequencies, with their corresponding amplitudes and phase angles, con- 
stitute the frequency contents of the original variable. 
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A newly developed technique of Fourier transform analyses [1] for 
digital computers, called the fast Fourier transform method, has made 
the use of these computers in frequency domain analyses very attractive. 
The method has been available in mathematical literature for some time 
[2] but it has been rediscovered by Cooley and Tukey only recently 
[3]. The Cooley-Tukey method reduces the number of multiplications 
required in calculating the frequency contents of a time function by a 
factor of y/N, as compared to the direct method of computation, where 
N = 2” is the number of discrete frequencies examined and y is an 
integer. The Cooley-Tukey method is primarily based on using binary 
logic for computation and for this reason the number of discrete 
frequency steps are taken in powers of two. 

In this chapter we will discuss the discrete Fourier analysis using 
matrices for computations, and we will proceed to describe the Cooley- 
Tukey computational process. The report also includes a digital com- 
puter program for obtaining the frequency contents of time functions. 


DISCRETE FOURIER TRANSFORM 


The exponential form of Fourier series of the periodic function F(t), 
with period T', is expressed as 


8 


Fi) = ) a,einet (4.1) 
and 
1 T/2 
a, = af F(t)e J"?! dt (4.2) 
( eens: 
where 


= angular frequency = 27f 
= frequency = 1/T 
= period 


= JT 


Pets na ES 
| 
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In Eqs. (4.1) and (4.2) it is assumed that the function is periodic with 
period T and the coefficient a, represents the integral of the function 
over a complete period. The corresponding Fourier transform pair of 
the function F'(t) is expressed 


Pip = z G(e@) eF?* da (4.3) 


co 


and 


os a a4 F(t)eJ@= dt (4.4) 
27 —oo 


where G(q) is the exponential Fourier transform of F(t). Equation (4.4) 
of the Fourier transform can be compared to Eq. (4.2) of the Fourier 
series if one assumes the period of the function F(t) to extend over the 
entire time domain. In practice, however, the function F(t) is given 
starting from time zero and extending over a span of time T, as given in 
Fig. 4-1(a). This will set the limits of Eq. (4.4) between zero and T. 
Given the Fourier transform G(qw) of the function F(t), Eq. (4.3), referred 
to as the inversion integral, will result in the function F(t). For the 
discrete representation of Eqs. (4.3) and (4.4), consider Fig. 4-1. The 
time span T of Fig. 4-1(a) is divided into K equal increments of At each. 
For convenience and efficiency of calculations, let us set K = 2”, where 
y is an integer. The angular frequency span wy, is divided into N equal 
increments of Aw each. Denote N in the binary form N = 2”*”! where 
y, is an integer. Note that although this definition allows the values of N 
and K to be different, in practice they are taken equal. The frequency 
interval Aw is defined as 


ho = 27 Af = = — (4.5) 


In the above, the value of Af is set at K/NT. This is needed to simplify 
the method of calculation. Equations (4.3) and (4.4) with the above 
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(a) 
ge ee ee eee tK-3 tK-2 tK-1 tk 
G (jw) 
(b) 
—| Nw 
(60) 
OQ @] W2 W3 Wy WN-3 ON-2ON-] ON 
figure 4-1. Discrete representation of F(t) and G (a). 
notation can be written in discrete form as follows 
N-1 
Fit) = do ) Glo,)eint (4.6) 
n=0 
K-1 
At -—-jw, t 
Glo, =  Y Flyedents (4.7) 
On k-0 


where the limits of the first sum instead of —N/2 to N/2 (corresponding 
to Eq. (4.3)) are set from 0 to N — 1 for computational convenience 
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As will be seen later, this changes the foldover frequency from zero to 
On / 9: The limits of Eq. (4.7) are taken to represent the available range 
of the variable F(t). Equations (4.6) and (4.7) represent the areas under 
the curves of Fig. 4-1. Using the following definitions, 


B= FQ) 
= tie 
2nnK (4.8) 
ie = nAw = 
NT 
pack ae “= 
k 


Eqs. (4.6) and (4.7) can be written 


N-1 
F. = Ao ) Ge(2ni/Nvink) (4.9) 
n=0 
K-1 
T eas 
ae > F’ o(-2mi/N) (nk) (4.10) 
27K k=0 
Denoting W = e~27//N, incorporating the constant T/27K into the func- 


tion F and denoting the product as F’,, Eq. (4.10) can be written in 
matrix form as 


eg ee) Se Oe ee ee ee | 


[G = Wink) [Fe 
a —_ Sal ae Se (4.11) 


(nx 1)(n x k)(R x VD 


where subscripts n and k represent the row numbers of column matrices 
[G] and [F], respectively, and the product nk in addition to being the 
power of W represents the row n and column & location of the W”* ele- 
ment. Before discussing the solution of matrix equation (4.11), con- 
sider the exponential function 
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ho Rami e cos (=) : jsin(=) (4.12) 
N N 


where 7 = nk is the exponent. Since the exponential function of Eq. 
(4.12) repeats itself in multiples of N, the values of J can be set 


l = nk 
(4.13) 
= nk modulus N 
= integer remainder after division of nk by N 
Under these conditions, / will assume values 0,1,2,3,..., N. It is further 
noted from Eq. (4.12) that 
ye yh 4 (4.14) 


and 


gore ef 


As an example, let us take the value of N = 8 = 23 and calculate the 
values of W! where 7=0,1,2,..., 8. Using the relation 


We - e(-27j/8) 1 ~ cos(") jsin(®) aie (4.15) 
4 4 


From this, the values of W’ can be calculated and plotted on the com- 
plex plane as shown in Fig. 4-2. It is noted that the foldover frequency, 
where W? and W-? are symmetrically located about the real axis, Occurs 
at 4 corresponding to the angular frequency of wN/2 or w4. The values 
of W'(1>N/2) are therefore the complex conjugates of the values of 
eos. 


MATRIX CALCULATION OF 


FOURIER TRANSFORMS 


In order to effectively explain the logic of calculating the values of 
the matrix equation (4.11), let us set N = K = 8 and write the expanded 
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Imaginary 
axis j 


Real axis 


Ww2 


figure 4-2. Powers of the exponential function W. 


form of this equation as follows: 


W 

; Gy WOL0,0) yyOCO,1) yy0C0,2) OC0,3) yOC0,4) OC0,5) yOC0,6) yO0,7)] | Fr 
nee WOCL,O) p11) 21,2) yy3C1,3) y4C1.4) WSUS) 6,6) W717] | 
2 1c. WOC2,0) 22,1) yyAC2,2) yO2,3) yO2.4) w22,5) wa2,6) w6(2,7)] | pr, 
3 1G, WOCS,0) (3,1) yy63,2) 13,3) wA3.4) W735) w23,6) W5E.7)| |p, 
4 1G, ~ | w0l4,0) y4(4,1) yy0(4,2) y4l4,3) pO(4,4) p4(4.5) pO(4,6) war4,7) F, 
5 |G, WOC5,0) y5(5,1) yy2(5,2) 75,3) yal5.4) WL5.5) W666) w35,7)] | p. 
6 |G, WOC6,0) yy6(G,1) yyAC6,2) 26,3) yOC6,4) yOV6,5) Ws6,6) 206.7) | | Fr 
7 |G WOC7.O) yTCT.1) wyOCT.2) 73) ACTA) WRI7.5) WA7.6) WL.7)} | Fr, 
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where the power of W, given outside the parentheses, is computed using 
relation (4.13). The row number n and the column number k are given 
within the parentheses (n,k). It is also noted that the W matrix is a 
symmetric matrix and its elements, calculated from Eq. (4.15), are com- 
plex numbers. 

Using the usual method of matrix multiplication, the G matrix can 
be calculated by multiplying the rows of the W matrix by the column 
of the F matrix. This will take N2 multiplications, or in this case, 64 
multiplications. In the following discussion, it will be shown that the 
number of multiplications can be considerably reduced if one uses some 
of the properties of the W matrix. The product of W and F matrices 
can be written as 


Column> 0 1 2 7 
+ WOR, + WOR, +... + WOR, 


a 


wir, + WF, fee ee WF, 


oO 
EY 


+ W°F, + WSF, + .-. + WF, 


m9) 


+ Wor, + WER, + -2 4 WR, (4.17) 
+W4F, + WOR, +... 4+ WF, 
+ WF, ~ WF, torre $ WF, 
+ WP, + WAFS +... + WEF, 
WF, + WSF, +... + WIR, 


6 ME Macca EAN Si Ha Ww Eira Re AL 
| 
i VOR ARS UNA, 9 CH RI RR 
> HN a = HACER 


“J 
| 

on) 
ae 


where W® of the first column of the W matrix of Eq. (4.16) is substituted 
by unity. The computation of G)G, ... G, of Eq. (4.17) can be ac- 
complished by initially setting their values equal to Fy, zeroth column, 
and adding the elements of the first column, and then the second 
column, and so on. It is noted that all of the W’s of the first column 
are multiplied by F,, and those of the second column by F’,, and so on. 
We further observe that in computing the elements WF, wi Ps 
W?F,, ..., Of the first column, the value of wN/2 _ w4 is equal to 
_w9 — -1, and the values of W after WN/2 are complex conjugate of 
previously computed values symmetrically located with respect to WN/ 2. 
For example, we only have to compute 


1 2 3 
Or Wr, Wer, nd WR, 
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since the rest of the values can be obtained from these, i.e., 


(4.18) 


where bars denote complex conjugates. 

From the second column of Eq. (4.17), it is noted that the values of 
the products repeat after the second Ww? F, is encountered. Thus, in this 
case, we only have to compute the values of wo F., and WF, since 


4 0 6 2 
wiP, = -WOF, and WF, = W?F, 


and the rest of the values W°F,, W7F,, W*F,, and WSF, are merely a 
repetition of the previously calculated set of products. The above 
procedure can be summarized into the following set of computational 
rules: 


1. Denote the number of time increments by Kk, frequency incre- 
ments by N, and the total time span by T. 

2. Obtain the values of W! where / = 0,1,2,..., N, using Eq. (4.15). 

3. Set the values of G,, G,,..-, Gy _ 1 each equal to Fy (Eq. (4.17), 
zeroth column). 

4. Compute the value of J, nk modulus N, where n is the row number 
and k is the column number of the elements of the W matrix of Eq. 
(4.16), starting with k = 1 and n = 0,1,2,...,N—1. Multiply the cor- 
responding W by F,. The first 1 will always be zero as it corresponds to 
the power of W in the zeroth row of the matrix of Eq. (4.16). When 
1 = N/2 obtain the complex conjugates of the previously calculated WF 
products, symmetrically located with respect to WN/2. If 1=0 the rest of 
the values of WF repeat. 

5. Add the values computed in step (4) to the existing values of 
GG,, =; Gxt 

6. Repeat steps (4) and (5) for k =2,3,4...,N—1. 

The logic of the above procedure can be programmed for digital 


computer solution. Once such program is given following the numerical 
examples. 
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The values of G calculated above will be complex numbers. The 
element Go will correspond to angular frequency » =0,G, tow = n Ao, 
and so on. It will be shown in the following discussion that the values of 
G symmetrically located with respect to Gy/o(= G4, for N = 8) in the 
column matrix of Eq. (4.16) will be complex conjugates, that is, 
G, = Gs, Ge = o- and Gy = Ge This fact will eliminate calculation of 
G’s beyond Gy//o. 

Consider the values of G, and G, calculated from matrix equation 


(4.16) and given below: 


2 4 6 
G5 = WOR WEP Oe WA Bae WE a OE, 
D) 4 6 
yr. Fe tM. F, 
0 6 4 2 0 (41) 
G, = Wor, + WF, + WAP, + WP, + WF, 


+ WF, + W4F, + WF, 


Since Wo = -W*= 1, the underlined values in the above equation will be 
real. From Fig. 4-2 it is observed that W? and W® are complex conjugates. 
Using this fact in Eq. (4.19) will show that G, and Gg are also complex 
conjugates. 
For future discussion, denote 
G. = a, + jb, 

where a, and b, are the real and imaginary parts of G,. The trigono- 
metric and the exponential forms of G, can be written as follows 


G, = |G,| (cosw, + jsiny,) 


ee 
where 
|G_| = absolute value of G, 
= Va,? & 3 e* 
b 
wv, = are tan ac 


an 
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The value of G, can be substituted for G(w,) in Eq. (4.6) to obtain 
the inverse Fourier transform. 


Example 4-1 


As the first example of obtaining the frequency contents by Fourier 
transform, consider the wave of Fig. 4-3(a). Setting the time and 
frequency steps at K = 16 and N = 16, the F matrix of Eq. (4.16) can be 
calculated by multiplying F(t) by T/2zK at each of the time increments 
At = T/K = 1. The sixteen values of W”* are then calculated using Eq. 
(4.12). From Eq. (4.5) the value of Aw = 27/16 = 0.39269. Having 
these values and using the logic described above, the values of G) through 
G,, can be calculated. 


F(t) 


Time t 


figure 4-3(a). F(t) vs. time, Example 1. 


Table 4-1 gives a list of the values calculated by the digital computer 
program to be described. The values of the function F(t) are shown in 
the first part of Table 4-1 while their Fourier transforms are shown on 
the lower part of the table. The fold over occurs at Gyo = Gg cor- 
responding to w = 7. It is further observed that the values of G’s sym- 
metrically located with respect to Gg are complex conjugates. 

For the purpose of discussion, let us write the Fourier series [4] of the 
time function of Fig. 4-3(a) 


ag = eed eee (4.20) 
ne 32 52 : 


3.2423 sinwt — 0.3602 sin3et + 0.1297 sindwi - --- 
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Freq. step n 


et ee oe 
A BRWNORKR OO COHN DN PRWNOK OO 


Table 4-1. 


0.392699 
0.785398 
1.1781 
1.5708 
1.9635 
2.35619 
2.74889 
3.14159 
3.53429 
3.92699 
4.31969 
4.71239 
5.10509 
5.49779 
5.89049 


Time step k 


© 


CONN NBRWHN = 


Solution of Example Problem 4-1 


Absolute value of 


|G 


4.18166 
0 
0.515635 
0 
0.230212 
0 
0.165452 
0 
0.165452 
0 
0.230212 
0 
0.515635 
0 
4.18166 


ay 
SD SO Oda ON cr GO WS 


rv 
—v 


ee 
Mn & W bP 


Function F (t) 


WNOre Or NW PWN K OO 


0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 


Imaginary part of 


G 


0 
—4.18166 
0 
0.515635 
0 
0.230212 
0 
0.165452 
0 
—0.165452 
0 
0.230212 
0 
“0.515025 
0 
4.18166 
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where w = 27/T = 0.392699. Consider the inversion Eq. (4.6) of the 
Fourier transform 


F(t) = Ao}(ay + jbg)[cosapt + j sinag fl 
+ (a, + jb,)|cosa@,t + jsino, t] (4.21) 
+ (dy + jbo) [cosw@jt + j sMaot| 


beef 


where a, and b,, as before, are the real and imaginary parts of G,. 
Substituting the values of a and b from Table 4-1, we get 


F(t) = 0.392699 }(-4.18166/) (cos, t + j sino, 0) 
+ (0.515635) (cos@.t + j sz?) 
+ (—0.230212)) (cos @,t + j sino, t) 
+ (0.165452j)(cosw,t + j sine, t) (4.22) 
+ (—0.165452)) (cos@gt + j SiN, t) 


+ (0.230212))(cos@,,t + j sinw,,?) 


reef 


Using the relations ©), = -©1, 13 = -3, and @,, = —@s, Eq. (4.22) 
simplifies to the form 


F(t) = 3.28419 sinw,t - 0.4050 sin agt + 0.1808 sinw, t as 


Comparison of the above and Eq. (4.20) indicates that although the time 
function of Fig. 4-3(a) was represented by only 16 points, the agree- 
ment between Fourier transform and Fourier series is acceptable. 


Example 4-2 


As a second example, consider the function F(t) of Fig. 4-3(b). In 
this case we again set N = K= 16 with T= 16. The results of calculation 
are shown in Table 4-2. The values of G given in this table should be 
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F(t) 


Time ¢ 


figure 4-3(b). F'(t)vs. time, Example 2. 


interpreted in conjunction with the inversion equation (4.6). Using the 
exponential form of G’s 


a = (es 


where |G_| is the absolute value of G,, and yw, is the phase angle, Eq. 
(4.6) can be written 


N44 
Fit) = Ao ), |G, ei ente + Hn) (4.23) 


n=0 


The above equation represents the value of the function F'(t,) for 
specific values of k. For example, for k = 0, t, = 0, we have 


N= 
F(0) = Aw x IG_ | el¥n 
n=0 
N-1 
= ho > (a, + jb,) 
n=0 


Substituting the values of a, and b, from Table 4-2, the above sum will 
result in the value of F(O) = 100. 

The amplitude Aw |G,| of Eq. (4.23) can be plotted as a function of w 
as Shown in Fig. 44. Note that in this figure the foldover frequency 


Freq. step k | Omega o 


pomed 
Owoonant WNK CO 


eon 
bestaoords 


jeden 
Mm BW bd 
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Table 4-2. Solution of Example Problem 4-2. 


0 


Time step k 


a a 
in wk Whe Oo OA) OO AR NM He © 


Absolute value of 


0.392699 
0.785398 


1.1781 

1.5708 

1.9635 

2.35619 
2.74889 
3.14159 
3.53429 
3.92699 
4.31969 
4.71239 
5.10509 
5.49779 
5.89049 


G 


127.324 


81.5801 
0 
28.6471 
0 
19.1414 
0 
16.2273 
0 
16.2273 
0 
19.1414 
0 
28.6471 
0 
81.5801 


a ee 
TonPtWNHNRK OO WON ANP WNK CO 


Function F(t) 


Real part of G 


127.324 


15.9155 
0 
$3.91>> 
0 
15.9155 
0 
15.9155 
0 
15.9155 
0 
15,9b55 
0 
15.9155 
0 
15.9155 


Imaginary part of 
G 


0 
—80.0126 
0 
seh S25 7 
0 
—10.6344 
0 
meee eS, 
0 
3.16579 
0 
10.6344 
0 
23.8192 
0 
80.0126 
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Amplitude 
Aw |G| 


Wg Wo W190 ©1] 012 @13 14 15 QO @1 2 03 BW4 W5 6 7 Wg 


figure 4-4. Amplitude-frequency plot of Fourier Transform of Example 2. 


occurs at wg and the values of ®) = —@7, @),) = —@g, .--, are plotted as 
negative frequencies. 


COMPUTER PROGRAM FOR THE CALCULATION 
OF FOURIER TRANSFORM 


The following computer program has been written in BASIC program- 
ming language [5] to implement the previously discussed calculations. 
The relation between the parameters of the program and those of the 
text are 


Program (symbol) Description Text (symbol) 
kl Number of time steps kK 
N Number of frequency steps N 
T Time span + 
Ti Time increment At 
R (J) Real part of W/ 

I(J) Imaginary part of W! 

F (J) Time function at j At 

G (J) Real part of G(a)) 

H (J) Imaginary part of G(wj) 
A Absolute value of G 


Nl Foldover value of G N/2 
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The loop 180-210 computes the required values of W. The loop 
220-250 reads into the computer the time function F(t) and incorporates 
the factor T/27K of Eq. (4.10). The loop 260-290 sets the values of G 
equal to F,. The loop 300-640 is used to compute the modulus of nk by 
N (this can be done more efficiently by binary arithmetic, see foot- 
note) and computes the required values of W multiplied by F' in each 
column of matrix W. When the value of L becomes equal to N/2 (or 
N1 in the program) statement 380, the product WN’?F’, is set equal to 
Wer, statement 420, and the complex conjugate values of previously 
obtained WF products are calculated, statements 480 and 490. When L 
becomes zero, statement 370, the values of W repeat and the instructions 
are transferred to statement 540. In the following loop 550-630 the 
calculated values of WF are incorporated into proper values of G. At 
the completion of statement 640, all the values of G’s are calculated and 
the rest of the program simply prints these values. 


FFFTTT 16:31 THURS. 04/11/68 


100 DIM R(100),1(100),F(100),G(100),H(100),U(100),V (100) 
110 READ K1,N,T 

120 LET P1=3.1415927 

130 LET P=(P1*2)/N 

140 LET T1=T/K1 

150 LET T2=P/T1 

160 LET N1=N/2 

170 LET T3=T1/(2*P1) 

180 FOR J=O TON 


The calculations of modulus of nk by N as given in statements 320-360 can be considerably 
simplified by using binary arithmetic. For example, for the row number n = 7 and the column 
number k = 6 the product nk will be 42, which in binary arithmetic is written 


5 4 a z 1 0 Power 
32 16 8 4 2 1 Powers of 2 
1 0 1 | 0 1 0 | Product nk in binary bits 
QO 43432) 
y = 3 bits 


The remainder is y binary bits on the right of the product nk, where y is obtained from 
N =2”. In the above, the remainder is equal to 2. 
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200 
210 
220 
230 
240 
250 
260 
270 
280 
290 
300 
310 
320 
330 
340 
350 
360 
370 
380 
390 
400 
410 
420 
430 
440 
450 
460 
470 
480 
490 
500 
510 
520 
530 
540 
550 
560 
570 
580 
590 
600 
610 
620 
630 
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LET R(J)=COS(P*J) 
LET I(J)=-SIN(P*J) 
NEXT J 

FOR J=0 TO K1-1 
READ F(J) 

LET F(J)=F(J)*T3 
NEXT J 

FOR J=0 TO N-1 
LET G(J)=F(0) 

LET H(J)=0 
NEXT J 

FOR K=1TO K1-1 
FOR J=0 TO N-1 
LET L=J*K 

IF L=N THEN 540 
IF L>N THEN 360 
GO TO 380 

LET L=L-INT(L/N)*N 
IF L=O THEN 540 
IF L=N1 THEN 420 
LET U(J)=R(L)*F(K) 
LET V(J)=I(L) *F(K) 
GO TO 510 

LET U(J)=-U(0) 
LET V(J)=-V(0) 
LET J3=J 

LET J3=J3-1 

IF J38<1 THEN 510 
LET J=J+1 

LET U(J)=U(J3) 
LET V(J)= -V(J3) 
GO TO 450 

NEXT J 

LET J1=J 

GO TO 550 

LET J1=J-1 

FOR J=0 TO N-1 
FOR M=0TO J1 
LET G(J)=G(J)+U(M) 
LET H(J)=H(J)+V(M) 
LET J=J+1 

IF J>N-1 THEN 640 
NEXT M 

LET:2=J-1 

NEXT J 
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640 NEXT K 

650 PRINT “TIME STEP”, “TIME”, “FUNCTION” 
660 FOR K=0TO K1-1 

670 PRINT K,K*T1,F(K)/T3 

680 NEXT K 

690 PRINT 

700 PRINT “FREQ STEP”,“OMEGA”,”“ABS VAL”,”“REAL PART” ,““IMAG PART” 
710 PRINT 

720 FOR J=O0TOW-1 

730 LET A=SOR(G(J)t2+H(J)t2) 

740 PRINT J,T2*J,A,G(J),H(J) 

750 NEXT J 

760 DATA 16,16,16 

770. DATA 0,1,2;3,4,3,2,1,0,-1,-2,-3,-4,-3,-2-) 

780 END 


COOLEY-TUKEY METHOD 


The Cooley-Tukey method of computing the G matrix of Eq. (14.16) 
is based on expressing the W matrix of the same equation in terms of 
products of y square matrices, where y is related to the order of these 
matrices N by N = 2%. In this method the number of time and frequency 
steps are the same and are equal to N. Consider the expanded form of 
Eq. (4.11) for N =4 


ap w9,0 wo.1 yo.2 ywo.3 Fy 
1,0 4 1,2 13 
Ve pag ee, . eA (4.24) 
= 2,0 a4 z2 23 
Gy| |W W W W - 
Ge yw3.9 wt w3.2 w3.3 F. 


Taking the products of the exponents of W, the term nk, and obtaining 
their modulus for N = 4, we get 


Go Fo 
G 0 1 2 3 
ae ee aS tel (4.25) 
2 2 
Gs WY ees. ASE 
3 2 1 
& ye We WIE, 
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Noting that the value of W° = 1, the W matrix of the above equation can 
be written as the product of two matrices (y = 2) as follows: 


Go t WwW © 2 ae 
2 
Saye ee Wiest 0) 1 ee (4.26) 
G, . DT £7) te 
G, 6 0 1 Wht 1 Bee 


The product of the above matrices will result in relation (4.25). Note 
that in Eq. (4.26) the order of the elements of the G matrix is changed. 
The product of the last two matrices of Eq. (4.26) is denoted as [F]! and 
is written 


Lists WBA 
0 
ie ea ee (4.27) 
LAL 
ee ee 


From the above, the elements of the [F]! matrix are 


1 0 
Fy = Fo + WF, 


0 
1 0 

F, = Fy + W F (4.28) 
1 Zz 

Fy = Fy + W F, 


. w2 
Pe Fee eS 


Having the elements of the [F]! matrix and using Eq. (4.26), the ele- 
ments of the G matrix are computed. 

The order of the elements of the G matrix in going from Eq. (4.25) 
to Eq. (4.26) is computed using binary arithmetic. The subscripts of 
these elements are first written in binary and then inverted to obtain 
the new order as follows: 
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Original Binary form Symmetrically : 
Subscript (y binary bits) inverted form Final subscript 
I O01 10 2 
2 10 01 1 
3 11 11 3 


As a further example for N = 8 = 2?, the above method will use 3 bit 
binary numbers resulting in the order of G’s as given below: 


Original Binary form Symmetrically 
Subscript (y binary bits) inverted form Final subscript 

0 000 000 0 

1 001 100 4 

2 010 010 2 

3 011 110 6 

4 100 001 1 

5 101 101 5 

6 110 O11 3 

7 111 111 7 


Multiplications and additions performed in obtaining the elements of 
[F]! matrix can be done by writing a computation tree and subsequently 
using binary arithmetic in computing the logic of operations. 

For the four-by-four W matrix of Eq. (4.26) consider the tree of 
Fig. 4-5 where the original F matrix is placed on the left. 


Column ——» 0 2 
- 

0 = Go 
= 3G. 
2 a 
3 = G3 


figure 4-5. Computation tree of the Examrple. 


Solid lines of Fig. 4-5 represent multiplications, and dashed lines repre- 
sent additions. The numbers in small circles are the powers of W. For 
example, e is obtained by multiplying F’, by W? and adding F, to it, 
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as given by Eq. (4.28). The logic of determining the powers of W, which 
are entered in the circles, and the multiplications and additions are done 
using binary arithmetic as given below. 


1. Assume N sampled values of F(t) where N = 2”. The integer y 
will equal the number of bits in the binary representation of N. 
2. Arrange for y columns of [F]!, [F]?, [F]°, ..., [F]” to be calculated. 
3. Let the numbers in the circles representing the powers of W be 
denoted by p. For row k and column 7 (starting with k and 7 equal to 
zero), the procedure for computing p is as follows: 
a. Write the binary number k. 
b. Slide this number y -— / places to the right filling the newly 
opened locations with zeros. 
c. Reverse the order of the resulting binary bits. The resulting 
integer is p. 
For example, for N = 8 = 2%(y = 3) the power in the circle of the 4th 
row (k = 4) and second column (J = 2) with y — 7 = 1 is computed as 
follows: 
a. The binary representation of k is 100. 
b. Since y — 7 = 1, shift k one bit to the right, resulting in the 
binary representation 010. 
c. Reverse the order of (b), 010, which in integer form equals 2, 
that is, p = 2. 
4. For multiplication and division denote the binary representation of 
row k by 


a fh = hak 


where ky, Ryy~-> are either zero or one. In column 1, node k has a 
solid line drawn to it from a node in the (J — 1) column having the 
same binary representation as k except the bit location Roy is replaced 
by one. For N = 16 = 24 (y = 4) the element in the third column (J = 3), 
fifth row (k = 5), will have a solid line drawn from the element in the 
second column having the row number determined by writing 


k = RgRok, ky = 9 and in binary = 0101 


Replacing k, element by 1 we get row number = O111 = 7, or row 
number seven. The dashed line comes from a node in the (/ — 1) 
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column with binary representation of k, where k.,_; is replaced by zero. 
In the above example this will result in k of 5. 

Using the above logic the computation tree of Fig. 4-6 for N = 16 can 
be constructed. 


DISCUSSION OF FOURIER TRANSFORMS 


Assuming that the function F(t) is real, which is usually the case, the 
discrete values of its Fourier transform can be obtained by multiplying 
W and F matrices of Eq. (4.16). For N discrete values of Fourier 
transform G, this will take N2 multiplications of complex numbers W’s 
by real numbers F’s. Taking advantage of the properties of the W 
matrix of Eq. (4.16) and the arrangement of product terms as given in 
Eq. (4.17), the number of multiplications can be considerably reduced. 
For example, for N = 16, the product matrices of Eq. (4.16) will require 
256 multiplications of complex numbers by real numbers. Using the 
matrix calculations described by the six-step logic will reduce the number 
of multiplications to about 85. Note that this method does not require 
multiplication of complex numbers by complex numbers which account 
for four real multiplications. 

The number of multiplications in the Cooley-Tukey method can be 
shown to be y% (where N = 2”). By this method for N = 16 (y = 4) the 
number of multiplications will be 64. As can be seen from the compu- 
tation tree of Cooley-Tukey method these multiplications involve com- 
plex numbers by complex numbers which should be counted as four 
real multiplications. 

Since a general expression for the number of multiplications required 
in the matrix method is not available, a quantitative comparison of 
matrix and Cooley-Tukey method cannot be made. It is estimated, 
however, that the real computational advantage of Cooley-Tukey method 
is to be gained when N assumes very large values (in the order of N = 28 
and above). 


PROBLEMS 


1. Show that G, and G, of Eq. (4.17) are complex conjugates. 

. Obtain the Fourier series of the function F(t) of Fig. 4-3(a). 
3. Obtain the real and imaginary parts of G, and G, of Table 4-1 by 
directly multiplying the proper elements of matrix equation (4.11). 
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Setting N = K = 8, find the set of values Gp, ..., G, for F(t) of 
Fig. 4-3(a). Answer: Aw = .392699, |Go| = |G»gl =9, |Gy| = 
4.34711, |G,| = .745846 

Repeat the above problem for N = K = 4 and find the values 
Gy, Gy, Go, and Gg. Answer: Aw = .392699, |Gp| = |Gol = 
0, |G,| = 5.09296 

Obtain the Fourier transform of 


Answer: Fy thru Fy = 2, 2, 2, 


F(t) 2, 2, 2s 
: ee 
Aw = .785393 
|G, |, = 1.66357 
: : Time? 2 -A@q| =;:880072 
<2. IGo| 7 Go| =) 


by matrix method using N = K= 8. 
Solve the above problem for N = k= 4. Answer. |G,| = 1.80063 
Obtain the Fourier transform of 


Answer: Fo thru F, = 8, 8, 8, 
F(t) 


: 8, 6, 4, 
2,0 
IGq| = 14.0056 
8 16 Time t IG,| = 4.61834 
IG,|.= 1.27324 


by matrix method using N = K= 8. 
Obtain the Fourier transform of 
Answer: N=K =8, Ao = .785398 
cae |G] = 4,45634...|G,! = 
.63662 
N =K =4, Aw = .785398 


| G,| =A.91l7Ea et = 
Time | ol prs 
: ! 1.27324 


by matrix method using N = K = 8 and compare with N = k= 4. 
Obtain the number of complex multiplications (complex numbers 
by real numbers) required for the computation of the G matrix 
of Eq. (4.17) for N = K=8 and N = k=4. 
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11. Obtain the Fourier transform of the function of Fig. 43(b) for 
N= k=8. 

12. Obtain Eq. (4.25) by performing the matrix multiplication of Eq. 
(4.26). 

13. Verify that the values of Go: Psy Gg obtained from Fig. 4-5 are the 
same as those of Eq. (4.25). 

14. Verify the additions, multiplications, and powers of W in the tree 
graph of Fig. 4-6. 

15. Write a tree graph for N = 8. 

16. Obtain the Fourier transform of Prob. 6 by the Cooley-Tukey 
method for N = k= 4. 

17. Obtain the Fourier transform of Prob. 8 by the Cooley-Tukey 
method for N= k= 8. 

18. Obtain the Fourier transform of Prob. 9 by the Cooley-Tukey 
method for N = k= 8. 
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5 NUMERICAL 
METHODS AND CALCULUS 
OF FINITE DIFFERENCES 


In the numerical solution of mathematical problems, calculations are 
performed at discrete time or space intervals. The numerical values of the 
solutions are obtained at these intervals. This is to be distinguished from 
classical solutions where a continuous solution in the form of an 
equation is derived and it can be evaluated for the given values of the 
independent variable. To illustrate this, consider the second-order 
linear differential equation 


—+y=0 (5.4) 


where ¢ is time and the initial conditions are y = 1, dy/dt = 0 at 
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t = 0. The solution of this equation is of the form (see Appendix) 


y = Asint + Bcost 


where A and B are constants, to be evaluated with initial conditions. 
This results in 


y = cost (3.2) 


This solution gives the value of y for any value of time t. On the other 
hand, the numerical solution of differential equation (5.1) is obtained 
by selecting a time interval At transforming the equation into difference 
form (which will be discussed later) and obtaining a recursive equation 
for y in terms of two values of y at preceding time intervals. Or, in 
functional form 


ee | — [Ys Ie op At) (553) 


the indexes.z and i— Irefer to values of y at times t and t._,(t,-t;_, = At) 
currently available and i + 1 refers to the value of y at t., 1 to be 
obtained. Thus, the application of the above relation together with the 
initial conditions will result in values of y at 0, At, 2At, 3At, ... and so 
on. It is important to note that, unlike solution (5.2), the value of y at 
t = nAt (where n is an integer) cannot be obtained unless the previous 
values of y are available. 

It is seen that the numerical calculations involved in obtaining the 
value of y from Eq. (5.3) at a given time ¢ are considerably more than the 
evaluation of solution (5.2). This is true in the above example since an 
analytical closed form solution exists for the differential equation (5.1). 
However, many engineering applications of numerical methods involve 
nonlinear differential equations, polynomial interpolation of tabulated 
functions, and the solution of transcendental equations where no 
analytical closed-form solutions exist. In these cases, solutions are 
obtained by numerical formulation of the problems and application of 
digital computers for evaluation of solutions. 

It should be also pointed out that in most engineering problems, even 
if closed-form solutions are available, they should be evaluated and 
plotted while the numerical methods result in solution already in 
tabulated or graphical forms. 
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Yi-2 
- 


Inn A x 0 Ax 2Ax 
figure 5-1. The function y = f(x). 


DIFFERENCE OPERATORS 


In numerical solutions, the difference operators are used in a manner 
similar to that of differential or Laplace operators (usually written as D 
or S) in the solution of differential equations. A complete familiarity 
with the difference operator is considered essential for an understanding 
of the underlying principles of numerical methods. 

In order to define various difference operators, consider the function 
y = f(x) described in Fig. 5-1 with equal intervals Ax for the independent 
variable x. 

The first backward difference of y, is defined as 


Vy; = 62 oes Kore’ (5.4) 


where V denotes the backward difference operator. 
The second backward difference can be obtained from the first: 
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Ved, Ng A = 93-4) 
Sa (5.5) 
es ee es = Hag) 


ee Fee 
Similarly, the third backward difference becomes 
Vey, = ¥; - 89;-1 + 8i-2 - Yi-3 (5.6) 
The first forward difference of y, is defined as 
Bes = 954° F; G7) 
where A denotes the forward difference operator. The second and third 


forward difference operators can be obtained from the first and second, 
respectively: 


A?y, = A(Ay,) 
= Ayi,1 - AY; 
= Bpee ine To; (5.8) 


Ae y, = A (A? y; 


a iss Fes OTe 
The first central difference operator of y, is defined as 
We tie Oe (5.9) 


where 8 denotes the central difference operator. Note that the first 
central difference involves the values of the function at half intervals. 
The second central difference operator is obtained from the first: 


2 
d“y, = Ody) = 819; 4 2 - Yi- 12) 


trea ee 541/91 ae Yi-vy2-v2) 


si Ve SUE V eet (5.10) 
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Similarly, third and fourth central difference operators can be obtained: 


°7) 
ie) 
Phe: 
II 


2 
5(6 y;) = 9543/2 — Wi+12 + 12 Si-az 


z : GS.4T) 

i= 5(8 y;) ere St a ee 
As a numerical example of the above equations, consider the calculation 
of difference operators of the equation y = x* as given in Table 5-1. In 
this table, the second and higher differences are obtained by arithmetical 
operations on the lower differences, for example, V2y = V(Vy)or V?y = 
5 —3=2. Only the value of 5y is given since Sy will require values of 
y at half interval points (i + 1/2). 


2 


Table 5-1. Difference Operators of y = x 


1 0 
2 2 0 2 
3 Zz Ps 0 2 
“ 2 0 2 0 2 
5 2 0 2 0 2 
6 2 0 = 0 Z 
i: 2 0 2 0 2 
8 2 0 3 0 = 
9 < 0 2 2 
10 2 0 2 
Hit 4-3 Z 0 


RELATIONSHIP BETWEEN DIFFERENCE AND 
DIFFERENTIAL OPERATORS 


Let us define the symbolic difference operator E as follows: 


By; = Via 

Beignicget (5.12) 
+1/2 

ee ee 


Using these relations, we can obtain difference operators V, A, and 6 
in terms of E 
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a 1 
ae ee (- Fy 


pce FE eee 


> ee “172, 4p -1/2 
Ss ee 


Equating both sides of these equations, we have 


ay oe oe backward difference operator 


(0.43) 


A=: fi forward difference operator 


6 - EV2_E-V2 © central difference operator 


The following algebraic relations also hold between the operators: 
Ak = kA 
Ek = kE (5.14) 
same for V and 6 operators 
EA = AE 
E" A = AE” 


where k and n are constants. 

The relation between difference operators. discussed above and the 
differential operator is obtained by writing Taylor’s series expansion of 
the function f(x + h) about x. This results in [2] 


? 2 ” ie 
f(x +h) = f(x) + Af (x) + — i 1 (5.15) 
! n! 


where primes denote derivatives with respect to x. Denoting the differ- 
ential operator D = d/dx and using the difference operator E, Eq. (5.15) 
becomes 


ee eee 
ad See sD ea fl 
Ef (x) Te ot + |r (5.16) 
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Equating both sides of Eq. (5.16) and noting the infinite series in the 
brackets represents the series for the exponential function, we have 


= ehD 
or 
inE 
bs = (5.17) 


The above formulas can be applied to obtain relationships between 
difference and differential operators. These relationships are used in 
the numerical solution of differential equations, interpolation and 
numerical differentiation, and integration of tabular values. 


RELATIONSHIP BETWEEN V AND D 


The backward difference operator V can be obtained in terms of the 
differential operator D with the aid of Eq. (5.13): 


= 4 te FP 


= 
E 


Taylor’s series expansion of e~”” will result. 


(5.18) 
2 3 

2 eae as. 4 
2! oe 


Similar expressions can be obtained for higher-order difference opera- 
tors: 
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VES ia Pe ST pene 


ie ae 
-1+(1- 32,29 -] 


1! 24 (5.19) 
| a ee Eo 
i 3! 
23 pe Mp. 2 apt 
12 


The product of Eqs. (5.18) and (5.19) will give the value of V2 in terms 
of differential operator D. 

Conversely, the values of differential operator D can be obtained in 
terms of the difference operator V by using the relation 


eee ae, 


(5.20) 
ie = Wp = ni - 
Writing the series for In(1 — V), we have 
2 3 4 
ieee (5.21) 
h 2 3 4 
Higher-order derivative operators D*, D°,..., can be obtained suc- 


cessively by forming products of lower-order derivatives. 
The relationships between V and D operators are summarized below: 


v4 3 
as oS ee ee (5.223) 
2! 3! 
ge ne pe 4 pt S:. (5.22b) 
12 
jo. j8ps = 2 98pt . "eee = 
9 zi (5.220) 


1 Ye Ve 5 ys 
p-1(0,2,0,%,... (5.22d) 
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ee L (v2, 9, Uo. S99... (5.22e) 
h2 ie 6 

a 1 (vs ,8v, 708...) (5.22f) 
h? 2 4 

a 1 (vs + QV? 4 ll v6 + -) (5.22g) 
h4 6 


The above relations give the infinite series representations of backward 
differences in terms of differentials and vice versa. In using these equa- 
tions in actual computation, however, we can only take a finite number 
of terms. Assuming that the series are convergent and that each term 
is smaller in magnitude than the preceding term (this is usually the case 
since the increment h/ is small and its powers increase), the first term 
dropped will represent the largest term neglected in the succeeding cal- 
culations. This term will give the “‘order of magnitude’? of terms 
neglected in the finite difference representation. It should be noted that 
in truncating an infinite series the sum of all of the terms dropped gives 
the actual truncation error. Since getting an expression for this sum is a 
formidable task in many cases, the first term dropped from the series 
is taken as the “‘order of magnitude” of the error introduced. For 
example, taking the first two terms of Eq. (5.22b) and neglecting the 
terms “i2h*D* and beyond will result in an error equivalent to the 
absolute value of this term. In addition, as will be seen in the discussions 
of this and the following chapters, since in many problems the value 
of the derivative of the variable D*y is also an unknown, this absolute 
value cannot be computed. For this reason we make another simplifica- 
tion in describing the error by stating that neglecting the third term and 
beyond of the series of Eq. (5.22b) introduces an “‘error of order h*”? in 
the resulting expression for V7. The error of order h* is written 
symbolically as error of O(h*). 

From Eq. (5.22) the derivatives of various functions can be obtained 
in terms of the available values of the function. Suppose that we want 
to obtain the value of D in terms of V’s with error of order h. From 
Eq. (5.22a) V involves terms of h2 and higher and since in Eq. (5.22d) 
this term, V7, is divided by h, neglecting it in Eq. (5.22d) will result in an 
error of orderh, as follows 
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V 1 
D = —, Dy, = —Vy, 
h h 
(5.23) 
5 2 eee) 2 ee 
St ee 
h 


where e = O(h) refers to the error of order h. If the error is to be of 
order h*[e = O(h?)], the V2 term in Eq. (5.22d) should be retained and 
the V3, V4,... terms dropped. This, using definitions of V, V2 results 


in 
1 v2 
Dy, ee ( + ey, 
h 2 


(5.24) 
e = O(h%) 


This result can also be obtained by setting V = 1 — E~! from Eq. 
(5.13) and raising to appropriate powers, that is, 


l| 
Pa 
aa ate 

| 

yy 

— 

f 

its 
mS ty 

o 
ul we) 
Wik ae 


T 
| 

© 
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The above relations can also be obtained for higher-order differentials. 
Table 5-2 summarizes the results. 


RELATIONSHIP BETWEEN A AND D 


The equations relating the forward difference operator A and the 
differential operator D are derived in a similar fashion as before. In 
this case from Eqs. (5.13) and (5.17), we have 
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Table 5-2. Relationship between Backward Difference Operator V and 
Differential Operator D 


ee - i+ A = ae? 


(5225) 
D = lad a ae eS 
h 
resulting in 
2 3 
4k a pe 
2! 3! 
2 3 
CoS pes ops 
at 3! 
and 
2 48 
ps ue eee (5.26) 
h ys 3 4 


Higher-order derivative and difference operators are obtained by 
taking various powers of Eq. (5.25). This results in the following second- 
and higher-order operator equations: 
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A? = 


ASS 


== 


b2D2 4 8D? .  fAD* = .. 
12 


np? . 2ntpt 4, 2 nS DF +... 
2 4 | 


he 2 4 
i ( aad , Le As 
h4 6 


(5.27a) 


(5.27b) 


(5.27c) 


(5.27d) 


(5.27e) 


Equations (5.26) and (5.27) can be used to find the differentials of 
functions in terms of available functional values. For example, neglecting 
errors of order A and higher, D? y, can be obtained from Eq. (5.27c). 
Dropping A® and higher-order terms from this equation and using Eq. 
(5.8), we have 


D2 y. 


l 


1 


= 7g i +2 =e a 


(5.28) 


Using this procedure a table of coefficients, corresponding to Table 5-2, 


can be calculated for forward-difference operators. 


This is given in 


Table 5-3. Note that the second relation above can be derived from the 
first by substituting A = 1 - E from Eq. (5.13) and using the definition 
of difference operator, that is, 


Dy 


t 


SE (a1 y. 


l 


p2 


- 1 - 26 + By, 


h2 


1 
Sees ee 
h2 
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Table 5-3. Relationship between Forward Difference Operator A and 
Differential Operator D 


=] | 
= SS - 
ze 3 
i =4 ] 


RELATIONSHIP BETWEEN 6 AND D 
OPERATORS 


The equations relating the central difference operator 5 and differ- 
ential operator D can be obtained using Eqs. (5.13) and (5.17). 


Se Ee wae” Bind (5.29) 
h 
In addition, let us define the mean operator p as follows: 


This will result in 


= tient) = te oe) 2 ei) 
. e (5.31) 
poly) = 5 it oer 


and 


seg eee ee y= (3.32) 
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Substituting the Taylor series expansion for sinh (hD), Eq. (5.31) becomes 


h?D? = A> p® 
= = 42 4 = 


pd = hD 
6 120 


S23) 


Using the expression for 6 from Eq. (5.29), squaring and substituting 
E = ewe have 


hD -hD 
2 a e + -€ 
Ge A 2a = (Aa - 1) (5.34) 


= 2[cosh(hD) — 1] 


Substituting the series for cosh (AD) results in 


44 6 6 
Sa ea (5.35) 
2 360 
The general expression for 5”, where n = 1,2,3,..., can be obtained from 
Eq. (5.29) 
oe ohD/2 = phDl2 "9 a hP 
(5.36) 
s" = 2 sinh WD 
2 


Using combinations of the above relations, we can obtain expressions 
for higher powers of central difference operators. For example, combi- 
nation of Eqs. (5.33) and (5.35) gives 


+ 


po? = (yd)(8%) = @ + 
6 120 
4n4 6 n6 (3.357) 
h2 D2 Fe Moe AvD 4 ) 
12 360 
h® D® : h’ D* 
4 40 


hn3p3 =A pd 
4 ae 


= A®p3 + ee 
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Squaring the first few terms of Eq. (5.35) will result in 


n&p® = =78p8 
Se +e a eee 
6 80 


64 = n*D4 + (5.38) 


The differential operators D can also be obtained in terms of central 
difference operators 6. 
For example, from Eqs. (5.31) and (5.32) we get 


7 3 5 
Tiree » sinh 1p5 ~ (2 — pow 2 pow - -) (5.39) 
h h 6 30 


where the right side of the equation is obtained from Taylor series 
expansion of sinh~!,5. Similarly, 


| 6 
pe 1(e_*,e_... (5.40a) 
h2 tS 
£6, 2...) (5.40b) 
h3 4 120 
6©& 7958 
o- 12.5...) (5.40c) 
h4 6 240 


Note that in the above relations, only odd differentials (D and D2?) 
contain terms involving p’s. 

Using the above relations, a table can be obtained relating differential 
and functional values. For example, the second differential D2, neglecting 
e - O(h2), can be obtained from Eq. (5.40a). From Eq. (5.38), 54 
involves terms of h* and higher; since this term (54) in Eq. (5.40a) is 
divided by h*,neglecting this will result in errors of order h? and higher. 
Using Eq. (5.10), this results 


Dey = =(0y, = = 0541 = 9y.+y¥,_,) ~e = O07) OAD 
h h 
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This result can also be obtained by substituting the value of 
6 = EV2 _ E-V2 from Eq. (5.13) and using the definition of operator 
E, that is, 


h2 
= 1 pve ae E-V2)’y, 
Lt 
p2 
i 7 
= —(E-2+E*)y. 
h2 


a ee ee 
h2 


Table 5-4 gives the relation between various orders of differential 
operators and central difference operators. 

The forward and backward differences are primarily used in incor- 
porating the initial and/or boundary conditions (as will be seen in 
Chaps. 6 and 7) in the finite difference representation of the differ- 
ential equation while the central differences are used for getting the 
general finite difference formulation. 

Example. As an example of application of the above equations, 
consider the numerical solution of the differential equation 


Table 5-4. Relationship between Central Difference Operator 6 and 
Differential Operator D 
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d*y dy 
ae 


Subject to initial conditions, y = 1, dy/dt = 0 att = 0. This equation in 
operator form can be written 


D*y, + Dy, + y, = 0 (5.43) 


where D = d/dt and the initial conditions are yo = 1, Dy, = 0. Using 
the central differences of order h*, from Table 5-4, we have 


meee) es ae 
ee OGY 


l n2 
(5.44) 
Nore io 
Dy, = eo . = 805 
2h 


where h = At is the assumed time interval. Substituting Eq. (5.44) in 


Eq. (5.43), we obtain a recursion relation for y, , ;, as follows: 


1 
ee 


= [(4 - 2h*)y, + (h - 2y;_4| (5.45) 


This equation gives the value of y,;,, provided that the two previous 
values of y are available, that is, y, and y;_,. To start the problem, 
two values of y will be needed. These will be obtained from initial 
conditions. Using central differences for the initial condition Dyg = 0, 
we have 


Dy, = ————— = Dc vou + % (5.46) 


Note that y_, in the above relation is inserted to make the derivative at 
Yo symmetrical. It actually does not exist in calculations. In addition, 
we have the initial condition given in the problem y, = 1. Using this 
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condition together with Eq. (5.46) and setting i = 0 in Eq. (5.45), we 
can solve for y,. Having the values of Yq and y,, Eq. (5.45) will give the 
value of yo(i = 1). Thus, the solution can be continued for i = 3,4,.. ., 
resulting in values of y at discrete time steps n At where n = 0,1,2,3,... . 

Mathematically, one of the important problems in the numerical 
solution of differential equations is choosing the value h = At. If At is 
too small (in addition to the problems of the stability of solution which 
may arise), a great number of iterations will be required to obtain the 
value of y for a given time. For example, if the value of y at t = 10 is 
required, a At = 0.01 will require 1,000 applications of Eq. (5.45), while 
a At = 0.1 will require only 100 applications of this equation for obtaining 
the desired value of y. On the other hand, if At is too large since the 
error is of the order of At?# (or A? in Eq. (5.44)), the error in each 
application of Eq. (5.45) may accumulate, resulting in large errors in the 
solution. Some theoretical background will be given for calculating the 
effect of At on the size of accumulated error in Chap. 7. However, in 
most engineering applications, the value of At is chosen by numerical 
experiments. This means that a value of At is chosen and the problem 
run on the computer and solutions obtained. The value of At is then 
varied and the solutions examined. In this way, a value is chosen that 
is a tradeoff between computation time and the desired accuracy of the 
solution. 


DERIVATIVE OF A TABULATED FUNCTION 


The derivatives of tabulated functions can be obtained with the aid of 
relations already derived. These derivatives are represented in terms of 
backward, forward, and central differences in Tables 5-2, 5-3, and 5-4, 
respectively. For example, the central difference approximation of 
order A? for the second derivative of a function is given in Table 5-4 as 


22 
oe ree, eee a 


Applying this formula to tabulated values of y = logx of Table 5-5 of 
page 210, we can obtain derivatives at various values of x. For x = 1.02, 
we have 

_ 0.0043214 -— 2(0.0086002) + 0.0128372 
pS ose (0.01) (0.01) 


~4.18 x 1079 


D? log x| (5.47) 
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INTEGRAL OF A TABULATED FUNCTION 


Using the symbol for the differential operator D, we can define an 
integral operator as the inverse of D: 


D-1 f(x) = Integral of f(x) (5.48) 


From this, 


ath 
3 f(x) dx = f(x) 
a D 


Riba: a He 
D 


a 


E-ligy 
D 


Similarly, for the integration interval between a and a + nh, we have 


et 
= f(a) (5.49) 


Since the integration interval in the above equation starts at a and 
increases to a + nh, it is reasonable to derive the integral in terms of 
forward differences. Using the relations 


in Eq. (5.49) and dividing the numerator by the denominator, we have 


at+nb 
f f(x) dx 


2 
= nsf fee , nan — 3) y2 (I ge |r 
ys 12 24 


(5.50) 
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For n = 1, neglecting terms involving A? and higher, we have 


at+h A 
f f(x) dx = h : 4 3] 10 


5.51 
|r aoe : is (5.51) 


II 


> Ifa) ree 


which can be recognized as the trapezoidal rule of integration. For 
n = 2, the term involving A? in Eq. (5.50) drops out. Neglecting terms 
involving A* and higher, we have 


a+2b A2 
3 f(x) dx = i(2 & ZA + “Vio (532) 


This can be readily evaluated by using the relation A = FE - 1. This 
results in 


at+2bh y) 
t ag ae i(2 oa ea 


h 


— + 4E + E*) f(a) (5.53) 


z = lfta + 4f(a +h) + fla + 2h)] 


which can be recognized as Simpson’s one-third rule of integration. 
Similarly, setting » = 3 in Eq. (5.50) and transforming in E operator 
form, 


at+3bh : 
f f(x) dx = = hi f(a) + Sf(a + h) + 3fla + 2h) + fla + 3h)] 
| (5.54) 


This equation is usually referred to as Simpson’s three-eighths ruie. 
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Integrals of tabulated functions can also be obtained in terms of 
backward and central differences. The central difference relations can 
be evaluated by writing the integral of the function f(x) ina symmetrical 


region 


at+nh 
z fix) dx = 
a-nbh 


at+nh 
foo |°"" fa 4 nh) ~ fla — nb) 
ee D (5.55) 


n =e 
See ee f (a) 
D 


For n = 1, this relation, from Eqs. (5.29), (5.30), and (5.39), can 


be written 


ath 
f fx) de = 
a-h 


II 


(EV2 _ f-V2)(RV2 , B-1/2) 


which, upon dividing, results in 


at+h 
af fds = 
a-h 


Neglecting the terms of 54 and higher gives 


ath 
7 f(x) dx = 
a-b 


which is Simpson’s 
form. 


f (a) 
D 
20H (a) 
D (5.56) 
oh be = 
Ou _ On uk + On — 
SS 
2 
gleoet og Acgtnk. cae (5.57) 
7 
2 
oh ( f a 
: (5.58) 


> [f(a — h) + 4f(a) + fla + h)] 


one-third rule of integration in central difference 
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For n = 2, Eq. (5.55) results in 


a+2b 1 pel a -1 
ef fin) be = (Gib eG +b) f(a) 
a-2h D 
(2h Sy) (82 + 2)" 
= 3 5 (5359) 
ee 
6 30 
: an(2 + 4a. T 54 5 
3 
Neglecting the terms of fourth and highe1 orders in 6, we get 
at+2h Dh 
+ f(x) dx = —[4f(a — h) — 2f(a) + 4fta + h)] (5.60) 
a-2bh 3 


Note that all integral formulas should give correct answers for f(x) = C = 
constant. This is usually used as a check of the integral formulas. For 


example, Eq. (5.60) for f(x) = C results in 


at+2h 2h 
i Cdk. = 46 = = 46. 2 oe (5.61) 
a-2h 3 


Integrals of functions can similarly be obtained in terms of backward 
differences. For this, consider integration over the interval a — nh toa: 


@ a 
[UC teoae "| f= fea = 
a-nb D =a D 


-n 
ae f(a) 
D 


n 
: es es f(a) 
D 


(continued) 


*This expression is obtained by writing 5 = E'’2 - E-!/2 and squaring 
8*¢- & +E !- 2. This gives E+ BE! = 82 + 2. 
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= ti ee a 


! ! 
2! 3! = 


(5,623 


Division of this expression will result in 


= n2 n? 
fled deen dein = Von 2 VS sft) 
a-—nb . 6 


This expression can be evaluated for different values of n. 


INTEGRATION FORMULAS BY 
INTERPOLATING POLYNOMIALS 


The integration formulas given above and additional integration for- 
mulas for the numerical solution of differential equations can be obtained 
by using polynomial interpolations. The basic idea is that a polynomial 
with unknown coefficients is assumed to pass through the sets of given 
points. Using the values of these points, the coefficients of the poly- 
nomial can be evaluated. These coefficients are then used to obtain 
integration formulas. 

Consider Fig. 5-1 of page 185 with variables x and y = f(x). Assume 
that the straight line 


y = Ax +B (5.63) 


passes through (x,, y,) and (x;, 1,9; ,). For simplicity, let us set x;= 0 
or equivalently take x, as the starting point and set the points (0, y,) and 
(Ax, y;, ,) in Eq. (5.63). This results in 


eae. 
a =< B 
(5.64) 
JP = A (Ax) SE B 


Solving these equations for A and B, we obtain 
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SS (5.65) 


Using these values, Eq. (5.63) becomes 


= P44 +y, (5.66) 
Ax 


The area under the curve from x = 0 to x = Ax is obtained by integration 
of Eq. (5.66). This gives 


Ax y: — yy. 2 
A = f ys = TL TE gyms « BEG, oy 
0 Ax Z 2 


(5.67) 
which is the trapezoidal law of integration. In a similar fashion, the 


Simpson rule of integration can be obtained by assuming the second- 
degree polynomial 


y = Ax* + Bx +C (5.68) 


passing through the points (—Ax, y,_ ,), (0, y,), (Ax, ¥;,,- Substituting 
these values in Eq. (5.68) and solving for A, B, and C, we obtain 


oe Lon eee 
Ape 532 35 sige ea ast eee es C= 4%; (5.69) 
2 (Ax)? 2Ax 
Using these values of A, B, and Cin Eq. (5.68) and integrating, 
Ax Ce 
Area = f y dx = = ee + 4y, + ¥;_) (5.70) 
Ax 


which is Simpson’s one-third rule of integration. 
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Expressions (5.67) and (5.70) give areas under the curve between 
points which satisfy expressions (5.63) and (5.68). It is sometimes 
desirable, as will be seen in predictor-corrector methods of solving dif- 
ferential equations (Chap. 6), to extend the range of integration using 
the same polynomials. For example, using Eq. (5.68) with the coeffi- 
cients (5.69), the area under the curve of Fig. 5-1 from -2Ax to 2Ax 
becomes 


2Ax Te 
| ee ee (5.71) 
=2hx 3 


This expression gives the area under the curve extending from (x, _ 5, ¥;_ 9) 
to (x; , 9, ¥;4 9) in terms of values of y extending from y,;_ , to y;, }- 
Differentiating Eq. (5.68) twice will result in 


ay 
= OS (5.72) 
dx? 


Qu 


Substituting the value of A from Eq. (5.69), we obtain 


fey Fj it M1 
dx? Ax? ee 


which is the central difference expression for the second derivative of y 
at x.. 
Differentiating Eq. (5.68) once, we obtain 


ae 2Ax + B (5.74) 


dx 
The value of this derivative at x = x, = 0 is obtained by substituting B 


from Eq. (5.69) and x = 0. This results in 


d see eee see 
Deg ED 75) 
dx 2Ax 


which is the central difference expression for the first derivative. 
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The integrals derived here, in addition to being used in obtaining the 
integrals of tabulated functions, are used in the numerical solution of 
differential equations, as will be discussed in Chap. 6. 


GREGORY-NEWTON INTERPOLATION FORMULA 


Consider the function f(x) tabulated from x =.a to x = b at incre- 
ments of Ax = h. It is desired to find the value of f(x) at x = a + nh 
where 7 can be a fraction. Using the relation 


f(a + nh) = E” f(a) (5.76) 


and the expression for E in terms of forward differences, we have 


E” f(a) = fla + nh) = (1 + A)" fla) 


E RA + “— b) A? +. ae a oa ae — AS 4 |r 
a! 


II 


ae Se 2 


The above equation can also be derived in terms of backward differences 
VY. For this we start from point b and write 


E" f(b) = f(b — nh) 


= (1 — V)" f(b) (5.78) 
: [ eke: Vv Z ae ‘ re 


Equations (5.77) and (5.78) are referred to as Gregory-Newton inter- 
polation formulas. Using these relations the values of tabulated functions 
can be obtained at any point within the interval by obtaining the required 
forward and backward differences. 

Example. Consider the table of log 9% for values of x between 1.0 
and 1.05 at increments of Ax = h = 0.01 to be given. It is desired to 
obtain logx at x = 1.024 by forward interpolation formula (5.77). 

The necessary calculations are represented in Table 5-5. Since the 
value of logx at 1.024 is desired, let us set f(a) = 1.02. The value of n is 


oe ADEA = 1028 <5 


0.010 
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Using this value of n together with proper values of Table 5-5 in 
Eq. (5.77), and neglecting terms of fourth order and higher, we have 


f(1.02 + 0.4 x 0.01) = 0.0086002 + 0.4(0.0042370) 


a (—0.0000409) (5.79) 


- 0.4(0.4 — 1)(0.4 - 2) 
6 


(0.0000008) 


From this 


f(1.024) = 0.0103000 


Note that the first two terms on the right of the equal sign in Eq. (5.79) 
represent the familiar linear interpolation terms. 


STERLING'S INTERPOLATION FORMULA 


Just as Gregory-Newton interpolation formulas are based on forward 
and backward differences, the Sterling interpolation formula is based on 
central differences. For the derivation of this, consider the Taylor series 
expansion of the function f(a + nh) 


2 5 
f(a + nh) = (: ake 7 2 PD? 2 -)fa (5.80) 
2! 3! 


where, as before, f(a + nh) is the value of tabulated function.at a + nh, 
h is the increment of independent variable x, and n can be a fraction. 
Substituting the values of D in terms of central difference operator 6, 
from Eas. (5.39) and (5.40) in Eq. (5.80), we have 


Combining powers of 6 results in 
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Table 5-5. Example of Gregory-Newton Interpolation Formula 


f(s) = logx Af (x) A? f (x) 


0.0000000 0.0043214 | —0.0000426 | 0.0000008 
1.01 0.0043214 0.9042788 | —0.0000418 | 0.0000009 
1.02 0.0086002 J.0042370 | —0.0000409 | 0.0000008 
1.03 0.0128372 0.0041961 | —0.0000401 
1.04 0.0170333 0.0041560 


0.0211893 


f(x) = logx 


0.0000000 


1.01 0.0043214 0.0043001 | —0.0000426 
1.02 0.0086002 0.0042579 | —0.0000418 0.0000008 
1.03 0.0128372 0.0042165 | —0.0000409 0.0000008 
1.04 0.0170333 0.0041760 | —0.0000401 


0.0211893 


=-1) pen is D) 54 


2 2 
f(a + nh) = E pene Ea ee See 
2! 3! 4! 


2 2 
amie EL ae = = sae + [ft 


(5.82) 


Equation (5.82) is the Sterling interpolation formula. This equation can 
be used for finding the values of the function at any point within the 
intervals by obtaining the required differences. 

Example. As an example of application of this formula, let us again 
consider the problem of evaluating log 1.024 from tabulated values of 
logx as given in Table 5-5. The necessary values to be used in Eq. (5.82) 
are given in Table 5-6. The values of » 6, 62,and u 6? are obtained from 
the following (previously given) equations: 


1 
p Oy; = ees tae de 


2 (5.83) 
O79 = Vege Oy + hg 


pd y, = (ud) (d2y,) 
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The value of n as obtained previously is 0.4. Using this value of n and 
the tabulated values of Table 5-6 and neglecting terms of fourth degree 
and higher in Eq. (5.82) results in 


f(1.024) = 0.0086002 + 0.4(0.0042579) + (— 0.0000418) 


(0.4)2 
2 


(5.84) 


, 0.40.16 = D (9 gggo008) = 0.0103000 


where f(a) = 0.0086002 is used as the starting point. This result agrees 
to seven decimal places with the result from the previous example. 


PROBLEMS 


1. Derive the expression for Mey, as given by Eq. (5.8). 
. Derive the expression for 54 y , as given by Ba-{5.,11): 

3. Obtain relations 5.22f and g for differential operators D° and D* in 
terms of backward difference operators. 

4. Obtain relations 5.27d and e for differential operators D? and D4 in 
terms of forward difference operators. 

5. Obtain relations 5.40b and c for differential operators D? and D* in 
terms of central difference operators 6 and pz. 

6. Suggest a method for solving the differential equation 


with dy/dt = 1 and y = 2 at t= 0, using central differences of order 
of h?. 

7. Using the tabulated values of tan x for x = 0.50 to 0.60 radians in 
intervals of Ax = 0.02 


x tan x 
0.50 0.54630 
0.52 0.57256 
0.54 0.59943 
0.56 0.62695 
0.58 0.65517 


0.60 0.68414 
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find the values of tanx at x = 0.55 radian using the Gregory-Newton 
forward interpolation formula and Sterling’s interpolation formula. 


8. Obtain the first and second derivatives of tan x at x = 0.54 using the 
above table and relations given in Table 5-4, for errors of the order 
h*. Compare these values to values obtained by differentiation of 
tan x. 

9. Using the tabulated values of tanx above, obtain the area under the 
tangent curve between x = 0.5 and x = 0.58 using trapezoidal and 
Simpson’s one-third (central difference formulation) rules. 

10. Take the equation 
y = Ax* + Bx +C 
and assume that it passes through the points (0, y;), (Ax, y,;, ,) and 
(2Ax, y;, 9). Evaluate the coefficients A, B, and C and obtain 
py 
Area = 7 y dx 
0 
11. Same as the above problem, but this time find 
3Ax 
Area = ys y dx 
-Ax 
12. Assuming the polynomial 
y = Ax? + Bx? + Cx + D 
passing through (0, y), (Ax, ¥;, 4), (2Ax, ¥;,5), and (3Ax, y;, 9), 
find the values of A, B,C, and D. Show that the value of A is 
proportional to the third derivative of y with respect to x, d? y/dx°. 
Compare the value of A with the forward difference Table 5-3. 
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6 NUMERICAL 
SOLUTION OF ORDINARY 
DIFFERENTIAL 
EQUATIONS 


The numerical solution of differential equations is accomplished by 
writing the differential equation in finite-difference form in terms of 
discrete time steps (At) and solving the resulting difference equation. 
This solution of the finite-difference equation will depend on the chosen 
value of time step At. In the limit when the value of At approaches 
zero, the solution of the finite-difference equation will approach the 
solution of the differential equation. It should also be noted that the 
analytic solution of a differential equation gives the value of the depend- 
ent variable x as a function of the independent variable t. This function 
can be evaluated for any value of the independent variable. On the 
other hand, the finite-difference solution of a particular differential 
equation will represent a set of values of the dependent variable x as a 


214 
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function of the independent variable t. The x’s will be obtained for 
values of independent variable 0, At, 2At, 3At,..., and so on. It will be 
seen that the solution x, at n At, where n is an integer, will depend on the 
values of x for (n — 1)At, (n — 2)At, ..., and soon. This being the case, 
the starting values as Higa Xoy +--+ should be calculated in order to be 
used in the general expression of the finite-difference representation. 


METHODS OF STARTING THE SOLUTION 


Taylor’s Method 


This method of starting the solution of a differential equation consists 
of representing the solution in the neighborhood of the initial values in a 
Taylor series expansion. This expansion will be an exact representation 
of the solution in the neighborhood of the initial conditions. Having 
this expansion, the values of the independent variable x can be evaluated 
for the few starting values of At. 

Consider the first-order differential equation 


x = ae f (x, t) (6.1) 


dt 


with the initial conditions x = x, at t = fp. Assuming the solution to be 
of the form 


<= ah (6.2) 


we can write Taylor series expansion of x about t¢ = ty as follows: 


x, tl ~ 12x 4 _;( SP ei 


where primes denote derivatives with respect to t. These derivatives are 
evaluated at {). The first derivative is given by x’ of Eq. (6.1), and the 
second derivative can be obtained by differentiating this with respect to 
t. Higher derivatives can be obtained in a similar fashion. That is, 
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x = f(x,t) = 81 (x, t) 


s de ia i a ye : 

fe A re aes, Se Eee = Ae 
dt oat. -de a ot Ox 
0g 08> 08 

ee he F Es sts - : x" = BAl,X, 20 (6.4) 
Ox ot Ox” 

ee ML, Bade = as x8 ~ 1) 


where g’s are functions of indicated variables. Starting with the first of 
Eq. (6.4), the values of x’, x”, x”, ... are expressed as functions of pre- 
ceding available values. For example, x” is a function of t, x, and x’, all 
of which are available from previous calculations. Note that in Eq. 
(6.3) the values of derivatives are evaluated at the initial point Xe and ty 
around which the Taylor series expansion is written. Having the values 
given by Eq. (6.4), the series of Eq. (6.3) which represents the solution 
of Eq. (6.1) in the neighborhood of the initial condition can be written. 

Example. It is desired to evaluate the value of the function x expressed 
by the differential equation 


dx 
dt 


for ¢ = 1.1, 1.2, 1.3, 1.4, and 1.5. The initial conditions are ¢ = 1, x = 1, 
Taylor series expansion corresponding to Eq. (6.3) can be written 


24x (6.5) 


fo 14 Geis 26 ee ee 
2! 3! 


where the values of the derivatives x’, x”, ... are evaluated at i> ie 
set of equations corresponding to Eq. (6.4) can be written 
= (P+ My yp yet = 2 
i =O a By Se 
xy = (6¢ + x4 4 = ll (6.7) 
ae = B+ wk”), , = 17 
xY = (x) 4 =.17 
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Thus the Taylor series expansion corresponding to Eq. (6.6) becomes 


meet tr 1902) + S(t —~ 1)2(5) + = (t Spay = ob Mt ~ 1)4(17) 


ob -y,- - -4)5 gy 4758 = 6.8 
ee 1) + st 1)® (17) + (6.8) 


The values of x can be evaluated using the terms given in Eq. (6.8) for 
the required values of t. These values are listed in Table 6-1. 

The solution of the differential Eq. (6.5) can be obtained by the 
methods described in the Appendix. This solution is given: 


x= Ac — ¢? — 312 ~ Gt - 6 (6.9) 


where A is the constant to be evaluated from the initial condition 
f= 71 g= £.  fhis results in A = 17/e, or 


Seta og Bf Gt —6 (6.10) 


The corresponding values of x fort = 1.1, 1.2,... are given in Table 6-1. 
Note that only the values of x obtained for ¢t = 1.5 differ in the fifth 
decimal place. 


Table 6-1. Required Values of x for the Example Problem 
of Eq. (6.5) 


From solution 
Eq. (6.10) 


From Taylor series 
Eq. (6.8) 


1.0 (initial 1.00000 1.00000 
value) 

1.1 1.22691 1.22691 

2 1.51585 L51585 

£3 1.88060 1.88060 

1.4 2.33701 2.33701 


Re. 2.90323 2.90326 
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Taylor Method for Higher-order and Sets of 
First-order Differential Equations 


The above method can be directly applied to higher-order differential 
equations. Consider the nth-order differential equation 
a x 


ee or Sti, ex" (6.11) 
: 


with the given initial conditions x,, x), xj, ---, x4 at 4). The Taylor 


series for the variable x of this equation around t, can be written 


a 3! 


+f 


Xq + 


X = Xy + fi>s ty) Xo + 


The values of the derivatives to xo 1 are already available from the 
initial conditions. Additional derivatives as required in the Taylor series 
expansion can be obtained by using the initial values and Eq. (6.11). 

In addition to this method, the nth-order differential equation (6.11) 
can be reduced to n first-order differential equations and Taylor’s method 
of starting the solution applied to the set of these first-order equations. 
The reduction to n first-order equations is accomplished by defining 
new variables 


x = 2,(t,x) 


z4 = Sag ee) = Z(t, x, 23) 


29 = x” = Z(t, X,Z1, 29) (6.17) 
2 eS ee 
= PU aed oes: & 4) 
where eee ee newly defined variables. Thus the nth-order 


differential equation (6.11) is transformed to n first-order differential 
equations (6.12). 

Taylor’s method of starting the solution of a set of simultaneous 
differential equations is similar to the previously given procedure. To 
illustrate the application of Taylor’s method to higher order and sets of 
simultaneous equations, consider the following examples. 
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Example. Consider the following second-order differential equation 


2 
eee ef 2 2 * Gis 
dt? 


with the initial conditions t = 0, x = 0, x’ = 1. The Taylor series repres- 
entation of the solution is written 


t? 13 
% = Xp + txp + — Xo + —Xp + ae (6.14) 
z' a 


The values of derivatives are obtained from the initial conditions and 
Eq. (6.13) as follows. 


Xy = 0 

Xo — 1 

eee ae 4 X69 = 0 

= = xo | (6.15) 

Yo 5 ee oe i. oe 

= Xo a 

Thus the series of Eq. (6.14) becomes 

3 5 4 

ee (6.16) 
a! 5! cg 


This solution can also be obtained by writing Eq. (6.13) as two first-order 
differential equations 


ee (6.17) 
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and writing Taylor series expansion for x and z. This solution, in addition 
to the values of the function x, will give its derivative z. Of course, once 
the function in the form of series (6.16) has been obtained, the derivative 
can be obtained merely by differentiation of Eq. (6.16). The series of 
Eq. (6.16) can be recognized as the series for sint. Thus, the values of x 
can be obtained for t = 0.05, 0.10, . . . using the first three terms of the 
series as given in Table 6-2. 

Differential equation (6.13) is recognized as the second-order differ- 
ential equation representmg the harmonic motion. The solution of this 
equation from elementary differential equation or the Appendix is 


x = Asint + Beost (6.18) 
Using the initial condition t = 0, x = 0, x’ = 1, this solution becomes 
x = sint (6.19) 


which is the solution represented by the series of Eq. (6.17). The values 
of x evaluated using Eq. (6.19) are also given in Table 6-2. 

Example. Consider Taylor series expansion of the solution of the 
simultaneous system of differential equations 


a = 47 4} 
dt 
(6.20) 
d2y 
pe See ee ge 
dt? 


Table 6-2. Required Values of x for the Example Problem 
of Eq. (6.13) 


Solution Eq. (6.19) 


| Three terms of 
series Eq. (6.16) 


O (initial 0.00000 0.00000 
condition) 

0.05 0.04998 0.04998 

0.10 0.09983 0.09983 

0.15 0.14944 0.14944 

0.20 0.19867 0.19867 

0.25 0.24740 0.24740 
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with the initial conditions ¢ = 0,x = 1, y= 0,y’= 1. Note that the 
first equation of the set is nonlinear as x appears to the second power. 
Equations (6.20) can be written-as a set of three first-order differ- 
ential equations by introducing z as the additional variable. This 
results: 


(6.21) 


Z2=2x+y-t 


with the initial conditions t = 0,x =1,y =0,2-= 1. The solutions 
expressed in Taylor series expansion about ¢ = 0 can be written 


2 ‘3 
X = Xp + Xp + a 
SL ge ee (6.22) 
P= Jes Wy I ae 
! 3! 
Zz ne te fae oe o. + 
Pe ay gy 8 


The values of the derivatives at t = 0 can be obtained by starting with 
Eqs. (6.21) as follows: 


Z = (x* 4 Aaa as 


es: = (x+y-t),_5 = l 


Aan + Y= 8 


(continued) 
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\" — [2(x’)? 4+ 2xx" |], 9 = 8 
i: AR Sole 1 
fe = ee = 4 
Z — (6.23) 
\" Na ee Be ee 2xx""), _ 9 = 34 
oe a 
|x ee ee 
Using the above values, the series of Eq. (6.22) becomes 
X = totale ae ee 
2 
2 2 A 
fe ee (6.24) 
2 6 6 
z= ieee ee Paes 
raat 8 


The initial values of x, y, and z can be evaluated from the above series. 


Euler Method of Starting the Solution 


The Euler method of starting the solution of a differential equation 
in effect uses the definition of the derivative as the ratio of two discrete 
intervals and proceeds to calculate values in the neighborhood of the 
initial starting point. This method, in many applications, is also used as a 
method for continuing the solutions. Consider the differential equation 


Oe ie yj (6.25) 


dx 


with the initial conditions x = Xo and y = yg: This equation can be 
written using the definition of derivative 


tee > Pee (6.26) 
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For small Ax’s 


dy Ax 
dx 


Ay 


Ie 


(6.27) 


Ie 


f (x, vy) Ax 


Thus, assuming a Ax and using the function f(x,y) from Eq. (6.25), the 
value of Ay can be calculated. The value of y for the next increment in x 
will be y + Ay. Although this method of computation is used for 
starting solutions, better results are obtained by using Taylor’s method. 
Note that the increment Ax in Euler’s method can be made small to 
improve the accuracy of the solution. 

Example. Consider the differential equation 


(6.28) 


with the initial conditions x = 0, y = 1. It is desired to find the values 
of y for x = 0.1 and 0.2 by Euler’s method. Let us take a Ax increment 
of 0.02 and write Eq. (6.28) in notation (6.27) 


2 
a ( : ‘ (0.02) 
ee > 
=0 (6.29) 


—0.01 


Thus the value of y for x = 0.02 will be y = 1 — 0.01 = 0.99. We continue 
the solution by writing Eq. (6.29) for x = 0.02 and y = 0.99 as 


2 
Ay = 2) (0.02) 
Ly pane 
— (0.9896 (6.30) 
1.99 


= —(.0099457 
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and the value of y becomes 
y = 0.99 -— 0.0099457 = 0.980054 


Continuing the above operation for x = 0.04, 0.06, . . . we obtain values 
of y given in Table 6-3. From this table the values of y for x = 0.1 and 
Q.2 can be obtained as required in the problem. Note that by assuming 
Ax = 0.1, Eq. (6.29) can be solved for Ay and, subsequently, y. This 
results in 


a 
Ay = (vs (0.1) = —0.05 (6.31) 
rey x=0 
i 
and 
y= 1.0: - 0.05 = 0.95 (6.32) 


This value differs from the more accurate value of y for x = 0.1 as 
given in Table 6-3. Continuing the operation with these values and the 
equation 


2 
Ay = Ee (0.1) (6.33) 
x=0.1 


Table 6-3. Values of y asa 
Function of x (Eq. (6.28)) 


1.000000 


0.02 0.990000 
0.04 0.980054 
0.06 0.970171 
0.08 0.960359 
0.10 0.950627 
0.12 0.940982 
0.14 0.931435 
0.16 0.921993 
0.18 0.912665 


0.903461 
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the value of y at x = 0.2 becomes 
y = 0.95 + Ay = 0.901795 


which again differs from the corresponding value given in Table 6-3. 

Note that if the differential equation contains fractions of the variables 
as in Eq. (6.28), the application of Euber’s method for starting the 
solution is simpler than Taylor’s expansion. The reason for this is that 
Taylor’s expansion requires first- and higher-order derivatives which are 
tedious to obtain in case of fractions. 


METHODS OF CONTINUING SOLUTION OF 
DIFFERENTIAL EQUATIONS 


Having obtained several starting values of the dependent variable, we 
can use the general finite difference expressions for continuing the solu- 
tion. The values of time or space increments, Atand Ax, can be changed 
during the methods of continuing the solution. For example, if during 
the initial transients, a higher solution accuracy is required, a smaller 
increment can be used and the increment increased during the latter 
portions of the solution. In this chapter, we will discuss the methods of 
continuing the solution for initial value problems while in the next 
chapter the method of solutions of boundary value problems will be 
discussed. 

The initial value problems are the ones in which the values of the 
dependent variable and its derivatives are specified at an initial point of 
the independent variable (these problems are also referred to as one-point 
boundary value problems). The initial point refers to the start of the 
problem. For example, in the differential equation 


+x = 0 (6.34) 
dt” 


the initial conditions can be expressed as the values x and dx/dt at some 
value of t = t). The solution of the differential equation will then apply 
for values of t > tp: Since Eq. (6.34) is a second-order differential 
equation, two conditions will be required to specify the constants of the 
solution. In the above case, the values of x and its derivative constituted 
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these conditions. This is to be compared to boundary value problems 
where the conditions at various values of the independent variable are 
given. In the case of Eq. (6.34) if the values of x at tyand x at t, > ty 
were specified, again the constants of the solution of the differential 
equation can be evaluated. Specifying these values converts the problem 
to a boundary value problem (also referred to as two-point boundary 
value problems). 

Most of the problems in electrical engineering are initial value problems 
as the independent variable of the differential equation is time. In 
these problems, conditions at time equal to zero are given and the 
solution is required for t > 0. However, in mechanical engineering, 
boundary value problems are encountered in elasticity and heat transfer. 
Examples of these will be bending problems in strength of material 
where the conditions at the end points are specified. 

In this chapter we will discuss the numerical solution of initial value 
problems, and in the next chapter we will extend these techniques to 
boundary value problems. 


Milne’s Method of Solution of Initial 
Value Problems 


Consider the differential equation 


y= 2 - fxy (6.35) 


x 


and assume that the first four points VV §v Vans corresponding 
tO X5,%X1,%9,%g are calculated by the previously given methods 
of starting the solution. Denoting the interval between x’s as 


Ax = %) — Xg = Xq — Xy = Xg - Xq = --+ Eq. (5.71) of Chapter 5 can be 
written 
2Ax 
rf as a ee) eee 
“28x 2 


Note that in the above expression y’ = f(x,y) is substituted for y of 
Eq. (5.71). The left side of Eq. (6.36) can be written 


2Ax Vit2 
d 
—-2Ax ax 


73-9 
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where i is used as an index. Using Eq. (6.37) in Eq. (6.36), we get 


AA 
Vina -Vi-g = =e a by + re) (6.38) 


It is more desirable to write the expression for y, , , in terms of previous 
y’s. This can be obtained by merely setting the index i = i — 1 in the 
above expression: 


ie Re os 3 ho) (6.39) 
3) 


Having the values of x, through x. and the corresponding values of y, 
through y,, the values of fo through f, can be calculated by substitution 
in f(x,y) of Eq. (6.35). Then by setting i = 3, 4,5, 6,... in Eq. (6.39), 
the values of y,, y5,-- for x4 = X3 + Ax, x, = x4, + Ax, .. .can be calcu- 
lated. Equation (6.39) is referred to as Milne’s predictor formula. 

In some problems it may be desirable to check or correct the solution 
by another integration formula. This is accomplished as follows. From 
Eq. (6.39) we obtain the value of y,, , corresponding to x;, ,. Setting 
these values in Eq. (6.35) will give f;, ;. Equation (5.70) of Chapter 5 
can be written in the form 


= Ax 
f yodxX. = = 


Ax 


where y is replaced by y’. Replacing y’ on the right of the above equation 
by f, we get 


A 
ee oes jet ee tied (6.40) 


Equation (6.40) is termed the corrector equation. The value of y;,4 
obtained from Eq. (6.40) is substituted in Eq. (6.35) and a new f,, , is 
obtained. This value is then substituted in Eq. (6.40) and a second 
iteration of y, , , is calculated. If this y;, , and the previous y, | , agree 
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to some specified accuracy, then y;, is calculated from Eq. (6.39); if 
not, the iteration given by Eq. (6.40) is repeated. However, in practice, 
a single application of corrector Eq. (6.40) is sufficient. It should be 
pointed out that in some applications the predictor-corrector method 
described above gives unstable solutions (see [2]). In addition, as seen 
from Eq. (6.39), Milne’s method requires four initial values computed 
by the previously given methods for starting the solution. 
Example. Consider the previously given differential equation (6.5) 


Se fa (6.41) 


with the initial condition t = 1,x = 1. Assuming that the first four 
values of x as a function ¢ are computed (Taylor series solution of 
Table 6-1), the value of f(x, t) in Eq. (6.41) can be obtained as follows: 


= 10 x, = 1.00000 f(x9,%)) = fy = 2-00000 


0 

t, = ll x, = 1.22691 f, = 2.55791 
to = 12 x, = 1.51585 f. = 3.24385 
tz = 13 x, = 1.88060 f, = 4.07760 


_ Having these values, we can apply the predictor Eq. (6.39) with i= 3 and 
calculate x , (same as yy of Eq. (6.39)) 


X= Xy + = Ag Sic (6.42) 
Assuming that At = 0.1, this becomes 


%, S104 (eS) x 4.07760 — 3.24385 + 2 x 2.55791) 
3 


= 2.33696 (6.43) 


which is the predicted value of x for t,= t,4 At= 13+ 02214, 
Setting this value of x, in Eq. (6.41), we get 


f,(x4,t,) = (1.4)? + 2.33696 = 5.08096 (6.44) 
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Using this value of f, in the corrector equation (6.40) with i = 3, we get 


‘ar ar = fp + Mf, + f4) = 1.51585 + 25 (3.24385 + 4 
3 


x 4.07760 + 5.08096) = 2.33702 (6.45) 


which is the corrected value of x,. Assuming that a single application of 
the corrector formula is sufficient (x, in Eq. (6.45) is close to x, in 
Eq. (6.43)), we calculate f 4 with the corrected value of x, 


fg = 41:4)? + 2.39702 =. 5.08102 (6.46) 


Now we return to the predictor equation (6.39) with i= 4 and calculate 
the value of x,. From this point the process is repeated, resulting in the 
values given in Table 6-4. In this table time represents the values of t, 


Table 6-4. Solution of the Example Problem Eq. (6.41) 


PRED X CORR X FUNCTION 
(Eq. (6.43)) | (Eq.(6.45)) | (Eq. (6.46)) 


ANALT SOL 
(Eq. (6.10)) 


1 l 2 1 
ia 1.22691 1.22691 ie Wee 2 1.22691 
2 1.51585 1.51585 3.24385 1.51585 
i 1.88060 1.88060 4.07760 1.88060 
1.4 2.33696 2.058102 5.08102 2.33702 
[5 2.90320 2.90326 6.27826 2.90326 
1.6 3.59994 3.60002 7.69602 3.60002 
if 4.45071 4.45080 9.36380 4.45080 
1.8 5.48210 5.48220 11.3142 5.48220 
ie, 6.72415 6.72426 i eel-ee) 6.72425 
2.0 8.21068 8.21080 16.2108 8.21079 
2.1 9.97970 9.97983 19.2408 9.97982 
ae 12.0739 12.0740 22.7220 12.0740 
ee 14.5409 14.5411 26.7081 14.5410 
2.4 17.4342 17.4344 31.2584 17.4344 
20 20.8135 20.8137 36.4387 20.8137 
2.6 24.7454 24.7456 42.3216 24.7456 
rae | 29.3039 29.3041 48.9871 29.3041 
2.8 34.5718 34.5720 56.5240 34.5720 
PA 40.6409 40.6412 65.0302 40.6412 
3.0 47.6137 47.6140 74.6140 47.6140 


55.6036 


55.6039 


85.3949 


55.6039 
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PRED X refers to values obtained from Eq. (6.42), CORR X refers to 
values obtained from Eq. (6.45), and FUNCTION refers to values 
obtained from Eq. (6.46). ‘The analytic solution which is the value of 
the solution given by Eq. (6.10) is listed in the last column of Table 6-4. 
As seen from the table the agreement between the analytical and finite 
difference solutions is excellent. 


Digital Computer Program for the Above Example 


The following digital computer program has been written in the 
BASIC computer language for a General Electric time-sharing computer. 
Although most of the statements of the program are self-explanatory, 
for further detail about this computer language the reader is referred 
to [3]. 


4 DIM T(100),X(100),Z(100),F (100) 

5 DEF FNF(T)=17*EXP(T-1)-Tt3-3*(Tt2)-6*T-6 
10 PRINT “TIME,’”“PRED X,” “CORR X,” “FUNCTION,” “ANALT SOL” 
11. PRINT 

20 LET T(0)=1. 

30 LET T(1)=1.1 

40 LET T(2)=1.2 

50 LET T(3)=1.3 

60 LET X(0)=1. 

70 LET X(1)=1.22691 

80 LET X(2)=1.51585 

90 LET X(3)=1.88060 

95 FOR I=0TO3 

100 LET F(I)=(T(1))t3+X(1) 

105 LET Z(1)=X(1) 

106 PRINT T(1),X(1),Z(1),F (1), ENF(T(1)) 

110 NEXT 

112 FOR l=3 TO 20 

115 LET T(I+1)=T(I)+.1 

120 LET X(I+1)=X(I-3)+((4*.1)/3)*(2* F(1)-F(1-1)+2* F(1-2)) 
130 LET F(I+1)=(T(1+1))t3+X(1+1) 

140 LET Z(1+1)=Z(I-1)+(0. 1/3) *(F(1-1)+4* F(1)+F (1+1)) 
145 LET F(I+1)=(T(1+1))t3+Z(1+1) 

150 PRINT T(1+1),X(14+1),Z(14+1),F(1+1),ENF(T(14+1)) 
155 LET X(I+1)=Z(I+1) 

160 NEXT 

170 END 


The program starts with the dimension statement 4, reserving computer 
memory locations for time T predicted x, X, corrected x, Z, and the value 
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of the function f(x,t),F. Statement 5 defines the function FNF (1), 
which represents the analytic solution of the problem given by Eq. (6.10). 
Statements 20 through 90 enter the four initial conditions. Statements 
95 through 110 calculate the values of the function for these initial 
values, and in addition set the corrected values of x and the predicted 
values of x for the four initial points equal, i.e., statement 105. State- 
ments 112 through 160 calculate the answers. Statement 120 is the 
predictor equation (6.39) and statement 140 is the corrector equation 
(6.40). Note that after values are printed out by statement 150 the 
values of corrected x’s are substituted for predicted values—statement 
ESS. 

The program computes values for 20 time steps (statement 112), as 
given in Table 6-4. 


Stability of Solutions 


In numerical computations, problems are formulated in terms of find- 
ing the values of dependent variable y,, in terms of previously computed 
values of this variable, its derivatives, the independent variable, and the 
increment h or At. Denoting the available values of variables by x, 
through x,,, the problem can be written in equation form 


¥, = FAX, Xo, ---, X,> h) (6.47) 
Since the previously computed values of x’s contain errors that may 
have been caused by, for example, truncation of significant figures, or 
the use of a particular difference scheme, the error will be reflected in 


succeeding values of y’s approximately according to the relation 


n af. 
ie ee (6.48) 


J 


where dy, and ox, are errors in the computed values of y; and x,. If the 
coefficients of Ox; terms, Of ,/ Ox ; are reasonably small, then Ox; errors 
will not degrade the solutions obtained for y;. In this case the solutions 
are termed stable. Note that these coefficients are also a function of the 
increment h. On the other hand, if the coefficients of ,/dx j are large, then 
seemingly small errors in 6x j values are magnified by these coefficients 


and repeated iterations will increase the error in y; values, rendering 
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the solution meaningless. Under these conditions the solutions are termed 
unstable. Note that Eq. (6.47) represents the particular finite-difference 
formulation of a problem and that the terms of Eq. (6.48) are a function 
of this representation. For this reason in a number of cases the actual 
solution of the problem may be stable but its finite-difference representa- 
tion may result in an unstable solution. Analytical methods of deter- 
mining the stability of a given finite difference representation will be 
discussed in the next chapter. 

Since the digital computers presently in operation carry a great number 
of significant figures, the computational errors and thereby the stability 
problems resulting from computations are usually small. In many engi- 
neering problems the inherent stability of finite-difference formulations 
is discovered when the problems are run on the computers and the 
solutions show behavior incompatible with physical and engineering 
reasoning. This method of analysis, of course, has no theoretical justifi- 
cation, but it reveals possible stability problems with the least amount 
of effort; theoretical analysis of stability criteria is extremely time 
consuming. Furthermore, in engineering applications the solutions are 
desired primarily during the initial transients, and most solutions that 
become unstable as the values of independent variable increases may be 
stable in the region of interest. 

To illustrate the above discussion we will analyze the errors in the 
finite difference formulation of the following example problem. 

Example. Consider the solution of the previously given example of 
Eq. (6.41). It is seen from Table 6-4 that the evaluated solution gives 
perfectly acceptable answers for the range of values of 1 < T < 3.1. Table 
6-4a gives the values evaluated to T = 10.0 under the identical conditions 
of time step and initial values. From this table it is noted that as T 
assumes larger values (although the solution remains acceptable), the 
error in the solution increases. The error is defined as the difference 
between the value given by the corrector formula and the analytical 
values obtained. 

From Table 6-4a it is seen that the percentage error (e.g., at T = 10.0 
the percent error is (1.75/136386) x 100 = 1.3 x 10°) remains very 
small. As discussed above, the error is partially caused by truncation of 
significant figures in the four initial values given following Eq. (6.41). 

As mentioned before, the stability of solution of predictor corrector 
methods has been studied extensively, as given in [2, 4]. In addition, 
[4] suggests a corrector formula developed by R. W. Hamming to replace 


Numerical Solution of Ordinary Differential Equations 233 


Table 6-4a. Error in the Computed Values Using Predictor-Corrector Method 


Value of solution 
Analytic Solution 


Absolute value of 
error* 


1.0 1.0 0 

2.0 8.21084 4.46 x 10° 
3.0 47.6143 47.6140 3.03 x 107 
4.0 199.455 199.454 1.32 x 10° 
5.0 692.173 692.168 4.94 x 10° 
6.0 2157.04 2157.02 1.71 x 102 
7.0 6320.34 6320.29 5.68 x 1072 
8.0 17884.9 17884.8 1.82 x 107! 
9.0 49644.8 49644.3 5.69x 107 

136388.0 136386.0 1.75 


*The values of errors were computed in the computer, which carries more significant 
figures than shown in the printout of the answers. 


Eq. (6.40). This formula in the previously given notation is as follows 


Vis = 2/99; - ¥¢-9 + BAR, 1 + 2F; - Ae (6.49) 


In [4] the errors developed using this equation and Eq. (6.40) are 
analyzed in the case of a particular example. It is shown that for this 
example the error in the solution using Eq. (6.49) as a corrector is less 
than using Eq. (6.40), which is Milne’s corrector equation. 


Adams’ Method 


Adams’ method of solving differential equations starts with the expres- 
sion for backward difference of a function in terms of differentials of 
the function, as given by Eq. (5.22a) of Chapter 5 


22 3ne 
ot ee Pee ee ae (6.50) 
: 24 


where fA is the increment of the independent variable. Substituting the 
values of differential operators D in terms of backward difference opera- 
tors (Eqs. (5.22)), we get 


234 Digital Computer Methods in Engineering 


rere rt h(is ts Sun, Sys, Dy. (6.51) 
a. 1s 8 


In obtaining the above expression the relation 


sue a 


is used. Note that in a given differential equation 


dy = f(x,y) (6.52) 


dx 


the value of Dy, = dy,/dx = f,. Using this, Eq. (6.51) becomes 


eee (eee bv 2084 ...)4 (6.53) 
8 


9 l 


This relation is Adams’ recursion equation for the solution of differential 
equations. Note that the values of y are based on the backward differ- 
ences of f,. 

In using the above relations let us assume that four initial values of 
y for x) through x, and the corresponding values of f, are calculated. 
That is, we have 


Xo Yo fy Vio Vat bs 


Sys. f55 nae V° fs 


where the differences (V’s) of f, are obtained from the given values. 
Having these values, y, can be computed by setting i = 3 in Eq. (6.53) 


Vighics 3 
et Ved Ng Se =i (6.54) 
4 3 (i ; 2D 37 5 3 
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After obtaining y, another row in the above table can be computed and 
iteration can proceed to yz, yg, and so on. Adams’ method also requires 
several starting values to continue the computations. 


Fox-Euler Method 


This method can be considered an extension of Euler’s method of 
starting the solution of differential equations as described previously and 
depicted in Eq. (6.27). This equation can also be obtained from the 
Taylor series expansion 


A ee 
ee te FY 


- aot (6.55) 


by neglecting terms involving second-order and higher derivatives. This 
results in 


Ws hy (6.56) 
dx 


which is identical to Eq. (6.27) with Ax = h. Suppose, in expression 
(6.55), instead of neglecting the second-order derivative term we sub- 
stitute this term with the value of y/ in terms of forward differences of 
yx , that is, 


ee (6.57) 
This will result: 
y.. 7. 
Vig ys Bis he ee (6.58) 


Or, finally, we get 


: ares : 
soe ae oo ts (6.59) 
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which is the Fox-Euler integration formula. Note that the second term 

on the right of the above equation is the average value of the derivative 

between y,,, and y; multiplied by h. Equation (6.59) can be used 

without the application of Taylor series for obtaining the starting values. 
Example. Consider the solution of the differential equation 


OY svg? ee ee a (6.60) 
dx 


with the initial conditions x = 0, y = 1. The analytical solution of this’ 
equation using the initial conditions is 


Wow Bet = x4 Oy — 2 (6.61) 


Using Eq. (6.60) in Eq. (6.59) results in 
h 
e3e1 = pens + ian HOP EY) (6.62) 


Solving this equation for y, , ,, we get 


= 1 hfe D 
re ey + 5 Ries + ag % ¥i)| (6.63) 


By setting the values of i = 0, 1, 2,3,...and assuming y = 1 for Xq = 0, 
the above solution can be evaluated for increments Ax = h of x. This 
solution is given in Table 6-5 for h = 0.5. 

For comparison, the analytical solution which is the evaluation of 
Eq. (6.61) is also shown. 

Using Eq. (6.56), which is Euler’s method for starting the solution, 
the values of y can also be evaluated as a function of x. This will give 


fe (6.64) 


which for the case of the example becomes 


7 y; + h(x;? + y;) (6.65) 


y (Ll + h) + he 
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Table 6-5. Solution of Eq. (6.60) 


y A 
(Eq. (6.63)) (Eq. (6.65)) 


y 
(Eq. (6.61)) 


0 1 ] 
OS 1.05135 1.05 1.05131 
] 1.10558 1.10263 1.10551 
1S 1.16311 1.15826 1.163 
a 1.22436 1.21729 1.22421 
as 1.28978 1.28016 1.28958 
3 1.35983 1.34729 1.35958 
35 1.43501 1.41916 1.4347 
A 1.51585 1.49624 1.51547 
AS 1.60288 1.57905 1.60244 
Be 1.69668 1.66813 1.69616 
32 1.79786 1.76404 1.79726 
6 1.90704 1.86736 1.90636 
65 2.0249 1.97873 2.02412 
- 2.15214 2.09879 215126 
mi 2.28949 222023 2.2885 
8 2.43774 2.36777 2.43662 
85 2.59768 2.51816 2.59644 
Fe 2.77019 2.68019 2.76881 
Bes 2.95616 2.8547 2.95463 
3.15655 3.04256 3.15485 


The solution resulting from this equation is also shown for comparison 
in Table 6-5. From this table it is noted that the values of y obtained 
from Eq. (6.63) more closely approximate the theoretical values of Eq. 
(6.61). The simpler equation (6.65) will give more accurate results if 
the time step is further decreased to, for example, 0.025. 


Solution of Simultaneous First-order 
Differential Equations 


The governing principles of the above solutions can equally well be 
applied to series of simultaneous first-order differential equations. As 
discussed previously and depicted by Eqs. (6.12), higher-order differ- 
ential equations can be written as a set of first-order simultaneous 
differential equations. Thereby, the methods of solution of simultaneous 
first-order differential equations is useful in solving higher-order equa- 
tions. 
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Consider the set of first-order differential equations 


: — f(x, y, 2) 
(6.66) 


e- e IY e) 


where the derivatives are with respect to x and the initial conditions are 
X= X%g,Y = Yq, and z= 2. Application of Adams’ method through 
Eq. (6.53) will result in 


ges ee 
| ed ee + Si 


zth(l+ c+ 2v24 Se, 
2 «AD 


(6.67) 


where h is the increment in the independent variable X ). After obtaining 
the first few values of y and z for initial x’s by Taylor’s method of 
starting the solution, Eqs. (6.67) can be applied much in the same way 
as for a single first-order differential equation. 

Example. Consider the previously given example problem of Eq. 
6.21)" 


ae ee Fa HD 
yi = z=, glz) (6.68) 


Ba Ke ya Sole, 9D 


where primes indicate derivatives with respect to t. Assume that the 
following initial values are given 


Pe to» Xo Yo Zo fo: Eq? Po (6.69) 


Neglecting terms involving V and V? in Eqs. (6.67), these equations can 
be written for x, y, and z as follows: 


higsi qe: Mgt NE; 
Fie t= Y; + Me; oo 
ee eee 
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Using conditions given in Eq. (6.69), the above equations can be solved 
for i = 0, resulting in x,, y,, and z,. Having these values, f,, g,,and 9 
can be calculated from Eq. (6.68). Then the application of Eq. (6.70) 
for i = 1 will result in the values of x5, y5,and zg. The solution can then 
proceed as before. 

Note that neglecting terms involving V and V2 in Eq. (6.67) reduced 
this formula to Euler’s relation (6.27). 


Milne’s Method for Second-order Equations 


In addition to the method of solution described above, in which a 
second-order differential equation can be reduced to two first-order 
differential equations and then solved, methods exist for the direct 
solution of second-order differential equations. The first method to 
be described is that of Milne. This method is similar to the previously 
described Milne’s method for first-order differential equations. 

Consider the second-order differential equation 


a = TT? (6.71) 


where primes indicate derivatives with respect to x, with the initial con- 
ditions x = Xp, ¥ = Yo: and y’ = yo. The values of Va» Vy. Yor Yor Vqo Va 
corresponding to x,,X5, and x, can be obtained using Taylor series 
expansion described for starting the problem. Having these initial 
values, fy, fy, fo: and f, can be computed from Eq. (6.71). 

Using Eq. (5.71) of Chapter 5 for y” instead of y, we get 


2Ax AN 
i ¥° Ge = ~ f, + 2f;_ 1) (6.72) 
- 3 


where the function of Eq. (6.71) is used in the right of this equation. 
Since 


2Ax 2Ax dy’ Vi+2 
f y" dx = ‘) dx = f dy 395.9 es pa (6a 
~2Ax - 24x dx y 


oa. th 
| eae 
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Eq. (6.72) can be written 
; ; 4 
Be. = Ying + Ohi - fet yd (6.74) 


Setting the value of index 7 = i - 1, Eq. (6.74) can be written in a more 
convenient form as 


; ; 4Ax 
Jo: Se Neck Weg td at ae (6.75) 


The above equation gives the value of y;,, in terms of four previous 
values of y’ and f. Having obtained the value of y;, , from Eq. (6.75), 
Eq. (5.70) of Chapter 5, Simpson’s one-third rule, gives 


Ag; : 5 
Poe ee Mere OOS (6.76) 


At this point we have all of the required values at i + | step, that is, 
X..4 = x; + Ax, y;,, from Eq. (6.76) and y; , , from Eq. (6.75). Having 
these values, f, , , can be computed from Eq. (6.71). 

At this point the corrector formula 


oi = Vit Ue + Mi + hee (6.77) 


can be applied to obtain a better approximation for y;,, than the 
previous value as obtained from Eq. (6.75). Note that the primary 
difference between values of y;, , obtained from Eqs. (6.75) and (6.77) 
lies in the fact that in Eq. (6.75) the value of y; , , is based on the values 
of f up to ith increment while in Eq. (6.77) the value of y%, , is based 
on the prior values of f as well as the current value, f,, ,. Therefore it 
is believed that Eq. (6.77) will give a better value for y;,,. Having 
this value of y;,,, Eq. (6.76) is reapplied and the value of y;, , com- 
puted. The procedure outlined following this equation may be reapplied 
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to compute another value of »;, , from Eq. (6.77), but in most cases a 
single application of the corrector formula (6.77) is sufficient. For the 
stability discussion of predictor-corrector methods the reader is referred 
to [4]. 


Noumerov’s Method of Solving Second-order 
Differential Equations 


When the first derivative term of a second-order differential equation 
is missing, Noumerov’s method of solution can be applied. Consider the 
following differential equation with derivatives with respect to x 


y” + f(xy = F(x) (6.78) 


with the y’ term absent. Substitution of y” in terms of its central differ- 
ence representation as given in Eq. (5.40a) of Chapter 5 will result in 


4 6 
( aoa tpl )y + h*f.y. = h°F. (6.79) 


' where h is increment in x. Multiplying both sides by (1 + 52/12), we get 


6 8 292 
(6 + cian = + “dy 2 bt: . ae ae 
12 


FAD - 15,120 
(6.80) 
252 
= h?F. + ae 
12 


Expanding terms involving 5% in terms of their arguments, we get 


D h 
og 8 yt IPE; + raucecare! = 2p et i719 


h2 66 1388 
Ba ir er (2 - +) 
ag eS te Nay ae 
(6.81) 
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Neglecting the last expression in the above equation will result in an 


error of 0(h°). Solving the above for y, , ,, we get 


1 2 2 
Lh /1y Ey 12 6 


12 (6.82) 
egg oem tas Lage 


which is Noumerov’s recursive expression for the solution of Eq. (6.78). 
Example. As an example of applying Noumerov’s method, consider 
the problem of solving the differential equation 


D 
See (6.83) 
dt? 


with the initial conditions t = 0, x = 1.0, and dy/dx = 0. Assume that 
the first few points of the solution are evaluated by Taylor’s method of 
starting the solution, as follows: 


t 0) x = 1.0 


f= 005 = x= Be 


II 


i= 010. x = ee (6.84) 


ix= 015 — x = 098877 


f= Ue = C500 


II 


i= V5 —-< = 0.90091 


Comparing Eqs. (6.83) and (6.78), we get 
ae eee eee eee (6.85) 


Using these relations in Eq. (6.82), we get 


2 2 
r= Es Ele (#- = (6.86) 
1 + (h2/12) 12 6 
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which is the recursion equation obtained from Noumerov’s method. 
Having two values of x, that is, x, and x, corresponding to é, and ¢,, the 
value of x. can be obtained from the above expression. 

Now again consider Eq. (6.83), but this time let us substitute the 
central difference representation for the second derivative term as given 
in Table 5.4 of Chapter 5 


Mey y — 2X; + Xj] 


= (6.87) 
h2 
From this we get 
ee 2 x; | (6.88) 


Note that the error in the above expression, from Table 5.4, is the order of 
h2 while the corresponding error Eq. (6.86) is in the order of h®. The 
recursion Eq. (6.88) also gives the value of x in terms of two previous 
values. Solutions given by Eqs. (6.86) and (6.88) are evaluated for At 
values of 0.1 and 0.20 and are given in Tables 6-6 and 6-7. The last 
columns of these tables give the values of the solution obtained from 


x = cost (6.89) 


which is the analytical solution of the problem. In Tables 6-6 and 6-/ 
(E-2) represents 102. From these tables it is seen that Noumerov’s 
method of Eq. (6.86) is closer to the analytical solution for both h = 0.1 
and 0.2. And, in addition, the central difference solution of Eq. (6.88) 
improves as the time step is reduced from 0.2 to 0.1. For example, the 
central difference solutions from Tables 6-6 and 6-7, for? = 1, are 


f= 62 i-= 1,0.- x = Sasso 


hR=@Di. i= 10 x = 0.540502 


Analytical solution ¢ = 1.0 x = 0.540302 


Thus the solution approaches the analytical solution as h assumes smaller 
values. 


244 Digital Computer Methods in Engineering 


Table 6-6. Solution of Eq. (6.83) for Time Step At = h=0.10 


Time? | = : = 
| (Eq. (6.86)) (Eq. (6.88)) (Eq. (6.89)) 


0 1 ] 1 

& | oS : 995 995004 
oz 980058 -98005 980067 
=. 955324 IS5299 955336 
4 921045 920996 921061 
BS) 877563 877483 877583 
6 825312 825194 825336 
es .764815 .764654 .164842 
8 .696677 .696467 .696707 
.O 621577 .621316 .62161 
1. .540267 Be Bee 240302 
1.1 453559 453187 453596 
i2 362319 361891 362358 
1.3 .267459 .266976 .267499 
1.4 .169926 . 169391 .169967 
Fe) 7.06955 E-2 7.01128 E-2 7.07372 E-2 
1.6 —2.92413 E-2 | —2.98669 E-2 —2.91995 E-2 
1.7 —.128886 —.129548 —.128844 
1.8 —.227243 —.227934 221202 
i ere PAR x, —.32404 = 32529 
2: —.416185 —.416906 —.416147 


Table 6-7. Solution of Eq. (6.83) for Time StepAt = h=0.20 


x x 
(Eq. (6.86)) (Eq. (6.88)) 


x 


(Eq. (6.89)) 


0 1 1 
ae 980080 980070 .980067 
4 .921067 920937 921061 
6 825345 824967 825336 
8 .696718 695998 .696707 
1. 540315 eg eo 540302 
Ez 362371 360813 362358 
1.4 .169980 .168004 .169967 
1.6 —2.91869 E-2 | -—3.15255 E-2 “2B ISOS E-2 
1.8 ~1227190 —.229794 ger ay St 2 
—.416137 —.418870 —.416147 
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The above solution can easily be programmed for a digital computer. 
The following is the program of the solution written in BASIC pro- 
gramming language. See [3]. 


5 DIM X(500),Y(500),Z(500) 
10 READ H,X(0),X(1) 

20 LET A=(((5/6)*H*H)-2)/(1+(1/12)*H*H) 
30 LET B=2-H*H 

40 LET Y(0)=X(0) 

50 LET Z(0)=X(0) 

60 LET Y(1)=X(1) 

65 LET Z(1)=COS(H) 

68 FOR I=1TO 20 

70 =LET X(I+1)=-X(I-1)-A*X(I) 
80 LET Y(I+1)=B*Y(I)-Y(I-1) 
90 LET Z(I+1)=COS(H*(I+1)) 
100 NEXTI 

110 PRINT “FOR H="H 

120 PRINT 

130 PRINT “TIME”,“NOUMEROV”,“CENTRAL DIFF”,“ANALYTIC SOL” 
140 FOR !I=0TO 20 

150 PRINT H*I,X(1),Y(I),Z(1) 
160 NEXT 

170 DATA .1,1.,.99500 

180 END 


In this program X is the solution obtained from Eq. (6.86) by 
Noumerov’s method, Y is the solution obtained from Eq. (6.88) by the 
central difference method, and Z is the analytical solution of Eq. (6.89). 
Statement 5 reserves memory storage for these variables. Statement 10 
reads the data of initial values. Statement 20 computes the coefficient 
of x. in Eq. (6.86). Statement 30 computes the coefficient of x, in Eq. 
(6.88). Statements 40 through 65 set the initial values of central differ- 
ence Y(0), Y(1), and analytical solution Z(0) and Z(1). Statements 68 
through 100 compute the Noumerov solution given by Eq. (6.86), the 
central difference solution given by Eq. (6.88) and the analytical solution 
given by Eq. (6.89). The rest of the statements print out the solutions. 
The computer output corresponding to this run is given in Table 6-6. 


Runge, Heun, and Kutta Integration Formulas 


These integration formulas are widely used in the solution of differ- 
ential equations with digital computers. Their primary advantage lies in 
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the fact that no special method is needed for calculating starting values as 
was the case with some of the methods presented. For the complete 
derivation of the formulas, the reader is referred to [5]. In this section 
we will give the general method of this derivation and will present several 
commonly used integration formulas. 

Taylor series expansion of the dependent variable y, in terms of the 
independent variable x; around the point x,, y, can be written 
2 cy , A 


9: - gis os (6.90) 


va, 


where Ax = x.,, — x, and the superscripts in parentheses specify the 
order of derivatives. Thus, having the values of x,, y, and the derivatives 
y\), v2), y,(3), ... the values of y, , , can be evaluated using Eq. (6.90). 
The value of x, , , is, of course, 


Migq = %, +h (6.91) 


where h is the increment in x. For a given differential equation 


Se a (6.92) 


dx 


the values of the derivatives can be obtained as follows: 


seg ela acer oes =— 
Ox oy dx : - 
Ms toe to Plt fol AG PG 


In these expressions x and y subscripts represent partial derivatives with 
respect to x and y variables. Evaluating these derivatives at x,, y, and 
setting in Eq. (6.90), we get 


Har 9p hb (6.94) 


Pe Pe a Se 
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where subscripts i refer to values at x,,y,. This formula represents the 
third-order approximation to the values of y as the fourth-order terms 
(terms containing h*) are neglected. 

Equation (6.94) is the basic equation in developing Runge’s integration 
formulas. Note that using only the first two terms on the right of the 
equal sign of Eq. (6.94) reduces this equation to Euler’s method. In this 
case, the error will be of order h? instead of h* of Runge’s formula. 

Using the definitions 


A’y = f(x,y) Ax = fh 

A”y = f(x + mh, yy + mA’y)h fee 

A”y = flx + Ah, yy + pAvy + (A - p) Aylh 
where m, A, and p are constants, it is possible to expand the above 
functions in Taylor series and sum up the resulting equations in the 
form 


_ Se eee 
= ct A’y = b A“y wg C Avy 


(6.96) 


where a, b, and c are again constants. Equating the terms of Eq. (6.96) 
with those of Eq. (6.95), after a considerable amount of algebra, relations 
between the constants m, A, p, a, b, and c can be obtained. Runge’s 
formulas are obtained by making certain assumptions in using these 
relations. For the complete derivation the reader is referred to [5]. 
Here, however, we will give several commonly used integration formulas 
obtained by the above procedure. 
1. Heun’s formula: 


mee.) 9; 


_— . (6.97) 
= i(A’y + 8A”), 
where 
Ay = f(x,y) h 
A”y = f.-(x+ith,y + LA’y)h 
l — 3 


se 
i 
II 


f x+ oh, y + ZA’y h 
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Kutta’s third-order rule: 


eae FS (A°y + 4A%y + A” y), (6.98) 


4, 


where 


me 
<< 
| 


= f(x, yh 

-? 1 1 ? 
Aya fet shy + sAyh 
Avy = fi + hye 2A y —Awh 


Note that this formula is analogous to Simpson’s one-third rule. 
Kutta’s fourth-order approximation (Kutta-Simpson one-third 


rule): 


Ay; = 2(A’y + 2A’y + 2N”y + AYy), 


ae 


where 
A‘y = f(x, y)h 
A*y = f.(x+ ay + LA‘y)h 
oe gee (6.99) 
Ww? a 1 i ” 
A - f(x + dh, y + LA’y)h 


Avy = f(x +h, y + A°Wh 


Kutta-Simpson three-eighths rule: 
Ay, = x (A’y + BA’y + BA”y + A’’y) 


where 
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ay fx, yh 

~~ 2 1 I , 

Ay fj(x+ghy+ gdh (6.100) 
Ay = [(x+2hy+ Ay - LA‘y)h 


A’y = flx+ hy + A’y - A*y + A’yh 


The error in using Eqs. (6.99) and (6.100) is of the order h° as 
opposed to order h* of Eqs. (6.97) and (6.98). In the derivation of the 
fourth-order equations an additional term containing h* was utilized in 
Eq. (6.94). 

As mentioned previously, in using Runge’s formulas no special method 
is needed in obtaining the initial values as was the case of predictor- 
corrector methods. In many applications the requirement of using a 
special starting method is considered a disadvantage. For this reason and 
because of its accuracy, Runge’s method is widely used in digital com- 
puter solution of differential equations. 

Example. As an example of Runge’s method of solution, consider the 
differential equation 


ee 0) =-0.5 (6.101) 


The analytic solution of this equation is 


ose .0,5e.°* (6.102) 


The program can be formulated for numerical solution by using, for 
example, Kutta’s third-order rule of Eq. (6.98). This can be done by 
assuming a value for Ax = h and systematically evaluating A’y, A”y, A” y 
of Eqs. (6.98) at x, =0, yp = 0.5. The resulting values are then used in 
the first of Eqs. (6.98) to obtain 


2S aguas Saas Ayo (6.103) 
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Similarly, for Yo. Y3, and so on. For comparison, solutions were also 
evaluated using only the first two terms of Eq. (6.94)—Euler’s method, 


J o5 ee y, + f;h (6.104) 


The results of these evaluations are given in Table 6-8 for A values of 
0.01, 0.05, and 0.1. From this table it is seen that Kutta’s formula 
(6.98) gives answers correct to four significant figures with an h = 0.1, 
while Euler’s method can be considered acceptable at h = 0.01. 

In practice, the value of the increment A to be used is arrived at by 
experimenting with the solution on the computer. In the case of the 
above problem, the problem is run with h = 0.1 and h = 0.05; if the 
answers do not change appreciably in the region of interest, the higher 
value of A(0.1) is chosen. The mere fact that the solution changes when 
the time or space increment is changed points to the fact that inherent 
errors existing in the finite difference representation of the problem are 
magnified by the value of the increment. For example, consider the 
values of the solution by Euler’s method of Table 6-8 at t = 1.0. It is 
noted that the solutions vary as h is decreased from 0.1 to 0.05 to 0.01. 
If this were the only method being used one might use A = 0.005 to see 
if the compounding of errors degrades the solution before deciding on 
the final value of h. However, in decreasing the values of h the reader 
should be aware of the problems of the stability of solution which 
might arise. 

Computer Program for the Evaluation of the Above Solution. The fol- 
lowing computer program in a version of FORTRAN programming 
language [6] was written to implement the above solution. In the 
program, Y2 refers to solution obtained by Eq. (6.98), Y3 by Eq. 
(6.104), and Y4 by Eq. (6.102). Statement 110 defines the function of 
Eq. (6.101). Statements 120 through 150 are constants. Statements 
170 through 220 are mechanization of Eqs. (6.98), while statement 
(230) represents Eq. (6.104). The analytical solution of Eq. (6.102) is 
given by statement 250. The results of the program are given in Table 
6-8. 


100 DIMENSION Y2(250),Y3(250),Y 4(250),X(250) 
110 FXY (X,Y)=EXP(-3.*X)-Y 

120 Y2(0)=.5 

130 Y3(0)=.5 
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Table 6-8. Kutta’s Third-order Rule in Solving Eq. (6.101) 


Time t Kutta Euler Analytic Increment 
equation (6.98) | equation (6.104) | solution (6.102) Ax =h 


h =0.01 


h = 0.05 


252 


140 
150 
160 
170 
180 
190 
200 
210 
220 
230 
240 
250 
260 
270 
280 
290 
300 
310 
320 
330 
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Y4(0)=.5 

H=.05 

DO101=0,199 

Al=! 

X(I)=AI*H 

D1Y=FXY(X(1),Y2(1))*H 
D2Y=FXY(X(1)+H/2.,Y2(1I)+D1Y/2.) *H 
D3Y=FXY(X(I)+H, Y2(1)+2.*D2Y-D1Y)*H 
Y¥2(1+1)=Y2(1)+(D1Y+4.*D2Y+D3Y)/6. 
Y3(I+1)=Y3(I+FXY(X(1), Y3(1))*H 
X(1+1)=X(I)+H 
Y4(1+1)=EXP(-X(14+1))-EXP(-3.* X(14+1))/2. 
10 

PRINT, “ TIME Y2 Y3 
PRINT 

DO 20 I=0,200,10 

PRINT,X(1),Y2(1), Y3(1),Y4¢1) 

20 

STOP 

END 


ROGUES 


Runge-Kutta Formulas for a Set of 
Simultaneous Equations 


Y4" 


As discussed in the beginning of this chapter, higher-order differential 
equations can be reduced to sets of simultaneous equations and these 
equations solved by various methods. One method in common use is the 
Runge-Kutta method, which is in turn an extension of the given formulas 


for the solution of first order equations. 
equations 

Bes [XX 9,2) 

dx 

si = g(x, ae z) 

dx 


Consider the simultaneous 


(6.105) 


For example, Kutta’s fourth-order approximation of Eq. (6.99) in this 
case becomes 


Ay. = x (A’y 4 9A 4 i 2A" 9s AD 


ae 


Az, = a (Az + OR 2s TA 2e ee 


(6.106) 
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where (for Ax = h) 


A’y 


f (x,y, zh 


A’z g(x,y, z)h 


ef 2 1 1" sae ee 
Bee es Gh Dy. e+ BA e)h 


Az 


1 me ee : ee 
g; x+lhy+td'y,z+1h2)h 
(6.107) 
ie 1 1 id zi ia 
Ay = f(x+shy+ 5h yz+ ghz h 


we = ; 1 : ere | a Sad 
Nz=8(x+ohy+5Ay2+ 5Azph 


Avy = f(x +h, y + Avy, 2+ A” 2h 


es 2 g(x = h, y s A’”’y, ae A” 2yh 


If these increments in y and z are computed in the order given, only 
previously computed increments will be needed at each step in compu- 
tation. 


Solution by Direct Substitution 


In a great number of problems it is possible to obtain solutions by 
directly substituting difference equations in place of differentials of a 
given differential equation. This was done in the case of the example 
problem in Eq. (6.87). In many closed-loop servo problems this method 
of direct substitution is the most practical way of solving the problem. 
In applying the method, care should be exercised to choose a time step 
which will give satisfactory solution. In engineering applications, this 
time step is chosen by experiments. That is, solution is obtained for a 
certain value of the time step and then this value of the time step is 
halved to see if the solution changes. If the solution remains practically 
the same, the original value of the time step is doubled and a solution 
obtained. The idea behind this is to obtain the largest practical value of 
the time step without detracting from the accuracy of the solution. 
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To illustrate the method of direct substitution, let us consider the 
following closed-loop navigation problem. 

Example. In this example we will illustrate the digital computer 
simulation of the proportional navigation which is often employed in 
missile guidance [7]. To derive the appropriate differential equations of 
this navigation scheme, consider the two-dimensional geometry of Fig. 
6-1. In this figure a missile centered coordinate system is used. The 
missile-to-target line-of-sight vector is designated by R and the closing 


AT 


d 
— Ro 
dt 
Target 


Missile target 
line of sight 


Missile 


figure 6-1. Two-dimensional diagram of proportional navigation. 


rate along this vector as R. The navigation scheme is based on the 
following relation 


es eee (6.108) 


which states that the acceleration of the missile perpendicular to R, Ay: 
should be equal to the acceleration of the target perpendicular to R, An, 
and equal to the rotational acceleration of line-of-sight R, A rp: From this 
figure it is noted that this will keep the line-of-sight R parallel and since 
R is the rate of decrease of the vector R, an intercept is assured. The 
line-of-sight acceleration Ap will consist of two parts, the acceleration 
perpendicular to R which will equal the rate of change of tangential 
velocity Rw where w is the angular rate, and the vectorial change of 
R(=R d6). This will result in 
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kh (6.109) 
dt dt 


Differentiating this expression and using the relation 


es (6.110) 
dt 
where @ is the angle of R with respect to some reference 
Ap = Roa + QeoR (6.111) 
Setting this value in Eq. (6.108) results in 
Ap - Ay = Ro + 2ok (6.112) 
Using differential operators D and D2, Eq. (6.112) becomes 
Ay - Ay = RD?0 + 2k DO (6.113) 
or : 
An 
ae ae (6.114) 
R D2 + 2k D 


This equation can be written in servo notation of [8]. The servo loop 
of Eq. (6.114) can be obtained by assuming that the missile acceleration 
Ay is proportional to the angular rate of 6 or Dé. In addition, let us 
assume the current value of range R as given by the equation 


R= Ry + ik (6.115) 


where R, is the initial value of R at t= 0 and t¢ is the time. Using this 
relation in Eq. (6.114) we can draw the servo loop shown in Fig. 6-2. 
In this figure at point a we have the numerator of Eq. (6.114) and at 
point b we have the value of 6. Assuming that the commanded missile 
acceleration is proportional to Dé, we have 


Ay = # DO (6.116) 
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—-Ayn + AT 


figure 6-2. Servo loop of proportional navigation 
problem. 


where k is the proportionality constant. Using Eq. (6.116) we get to 
point c of the loop of Fig. 6-2. From this point we get back to a and the 
loop is closed. Note that integrating acceleration A,, twice will give the 
position of the missile, P of Fig. 6-2. 

As a numerical example of simulating the loop of Fig. 6-2, consider 
the following values and Fig. 6-3. A missile with a velocity of 1,500 fps 


Vo = 1,500 ft/sec Vy = 500 ft/sec 


Missile e———____________._ Target 


= 000 fh ae 


figure 6-3. Numerical values of the proportional 
navigation problem. 


is fired at an approaching target with a velocity of 500 fps at a range of 
20,000 ft. At this instant the target maneuvers with a vertical accelera- 
tion of 32 fps, keeping its horizontal velocity constant. Assuming that 
the missile horizontal velocity remains constant, what are the missile 
vertical accelerations, using proportional navigation? To write the digital 
simulation of the problem, we use the central difference methods of 
Chap. 5. Let us denote 


3 Ayes (6.117) 


Equation (6.114) can be written, using central differences for the deriva- 
tives as 


a ee = 2S a 
(2, +) ——___- , tu, 6.118) 
(At)2 2At 


Numerical Solution of Ordinary Differential Equations 257 


This equation can be solved for 6, , , in terms of 6,and 6,_,. Having the 
value of 6,,,, point 6 of the loop of Fig. 6-2, the value of missile 


acceleration A,, can be obtained from (6.116): 
0; ss 
a (6.119) 


The position of the missile P can be obtained from 
D2P.= Ay (6.120) 


or, in finite difference form, 


oe ee See 


24 
= 2 (Ay), (6.121) 


This equation can be solved for P. , , in terms of P, and P,_,. Thus the 
loop of Fig. 6-2 can be started by setting initial values of 0, A,,, and P 
equal to zero and introducing target acceleration A;. The positions of 
the missile and target with respect to the initial missile position can be 
obtained by using the following relation: 


Missile (horizontal) = Vjt = 1,500¢ 
Target (horizontal) = Ry —- V,t = 20,000 — 500¢ 

(6.122) 
Missile (vertical) = P (Eq. (6.121)) 


Target (vertical) 


i 2 2 


The following digital computer program has been written in BASIC 
programming language (see [3]) to accomplish the above procedure. 
In the program, the following notations are used: 


L = angle @ in radians 
A missile acceleration, ft/sec? 
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P = missile vertical position, ft 

RO = R, initial value of R, ft 

V1 = target horizontal velocity, ft/sec 

V2 = missile horizontal velocity, ft/sec 

K = proportionality constant of Eq. (6.116) (ft/sec? )/(rad/sec) 
Al = target vertical acceleration, ft/sec? 

T2 = time step size At, sec 


Rl = R or closing rate =—(V1 + V2) 


10 DIM L(500),A(500),P(500) 

20 READ RO,V1,V2,K,A1,T2 

25 LET R1=-(V1+V2) 

30 PRINT “RZERO”RO,”TGT VEL”V1,”MSL VEL” V2 
35 = PRINT “K”K,“TGT ACC” A1,"DEL T”T2 

36 PRINT 

40 LETL(O)=0 

41 LETL(1)=0 

45 LET A(0)=0 

46 LET A(1)=0 

50 LET P(0)=0 

51 LET P(1)=0 

65 LET 1I1=RO/(-R1*T2) 

70 FOR I=1T0O11-2 

75 LET T=!*T2 

80 LET U=A1-A(I) 

90 LET R2=RO+T*R1 

100 LET L(I+1)=U*T2*T2+L(1) *(2*R2)+L(I-1)*(R1*T2-R2) 
110 LET L(I+1)=L(I+1)/(R2+R 1*T2) 

120 LET A(I+1)=((L(1+1)-L(1))/T2)*K 

130 LET P(I+1)=T2*T2*A(I)+2*P(1)-P(I-1) 

140 NEXT! 

145 LET P(I11)=T2*T2*A(I1-1)+2*P(11-1)-P(1 1-2) 

150 PRINT “TIME”,“TGT POS H’’,“TGT POS V”, 

160 PRINT “MSL POS H”,”“MSL POS Vv” 

170 FOR J=0 TO 11 STEP 20 

180 PRINT J*T2,RO-J*T2*V 1,(A1/2) *(J*T2)t2,J*T2*V2,P(J) 
190 NEXT J 

200 LET J=I1 

205 PRINT 

210 PRINT J*12,RO-J*T2*V 1,(A 1/2) *(J*T2)t2,J*T2*V2,P(J) 
220 PRINT 

230 PRINT “TIME”,“ANGLE”,“MSL VER ACC” 

235 FOR J=0 TO 11 STEP 20 
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240 PRINT J*7T2,57.3*L(J),A(J) 

250 LET J=!I1 

255 PRINT 

260 PRINT J*T2,57.3*L(J),A(J) 

270 DATA 20000,500, 1500,20000,32,.025 
280 END 


Statement 10 of the program reserves storage locations for L, A, and P. 
Statement 20 reads the data as given inStatement 270. These values are 


RO = 20,000 ft 

V1 = 500 ft/sec 

V2 = 1,500 ft/sec 

K = 20,000 (ft/sec? )/(rad/sec) 
Al = 32 ft/sec? 

T2 = 0.025 sec 


Statement 25 sets the value of rate of change of R equal to the sum of V1 
and V2 with a negative sign (since R decreases as the missile and target 
approach each other). It is assumed here that the angular change of 
vector R is small and R remains practically horizontal during the flight. 
Statements 30 through 36 are print statements which write out the 
input data. Statements 40 through 51 set initial conditions equal to zero. 
Statement 65 computes the upper bound of time index I. This value will 
be equal to the initial range divided by distance moved at each time step. 
Statements 70 through 140 are the main loop of the program which com- 
pute Ld+1) = 6.,, from Eq. (6.118), Ad +) = (Ay)... from Eq. 


(6.119), and P+ 1) = Pa from Eq. (6.121). Note that the last 
position of the missile P(J1) is computed in statement 145, since at the 
last point the missile-to-target angle L according to statement 110 will 
go to infinity as the denominator of L goes to zero. Statements 150 
through 190 print time, horizontal target position (TGT POS H), vertical 
target position (TGT POS V) and horizontal and vertical missile positions 
(MSL POS H and MSL POS V), respectively. Note that in print state- 
ment 180 Eqs. (6.122) are used. Since statement 170 is indexed every 
20 time steps there may be cases where the values at the last computed 
time step will not be printed. For this reason statement 210 prints the 
last computed values. 
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Statements 230 through 260 print out time, angle L = 6 (in degrees), 
and missile vertical acceleration A (MSL VER ACC). 

The output listing of the computer run is given in Table 6-9. (This 
output is obtained by combining the outputs of statements 170 through 
260.) A graph of the horizontal distance and vertical distance with 
time ticks is shown in Fig. 6-4. 


Nonlinearities and Random Inputs 


In digital computer programs nonlinearities describing the limits of 
variables can be entered through logical instructions. For example, if it 
is desired to limit the missile acceleration A, of Fig. 6-2 at point c toa 
predetermined value of +k,, this can be entered as a logical operation at 
point c and correspondingly in the computer program. The following 
flow chart describes this. 


lA, > No 


Random variables can be introduced equally well into digital computer 
programs. Almost all of the digital computer installations have built-in 
routines for the generation of random numbers y, with uniform 
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Missile acceleration 


Elevation, ft 


Missile track 


Missile and target acceleration, ft/sec2 


8 10 {2 14 
Distance, 1,000 ft 


figure 6-4. Graphical representation of solution given in Table 6-9. 


distribution, between zero and one. For this distribution mean and 
standard deviations are given as 


Mean = ply) = 5 
(6.123) 


Standard deviation = o7(y) = 


II 


Since many of the errors in engineering systems can be represented by 
normally distributed white noise,* it is desirable to obtain random 
numbers with this distribution. Using numbers y from a uniform distri- 
bution (0, 1), it is possible to obtain numbers v from a normal distribution 
N(p,o), where p is the mean and o is the standard derivation. To 
accomplish this the following procedures are used: 

1. Approximate method—From the central limit theorem [9] 
using Eqs. (6.123) we can obtain the following relations 


ye -< 
=) 2 (6.124) 


*The term white noise is used when errors are independent of each other and are not time 
correlated. 
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For most applications k > 6 produces a distribution of v closely approxi- 
mating a normal distribution N(y,o). From Eq. (6.124) k = 12 isa 
particularly desirable choice, since the division by the radical is eliminated 
and Eq. (6.124) reduces to the form 


12 
(Yoon (6.125) 
i=1 


In this expression, y,, as before, represents a random number from a 
uniform distribution (0, 1). 

2. Exact method—Using statistical analysis (see [10] ), it is possible 
to obtain two random numbers from a normal distribution N(O, 1) by 
using two random numbers from a uniform distribution (0, 1). Assuming 
that the random numbers are y, and y,, we have 


z, = V-2|ny, cos (275) 


(6.126) 


ig = V-2Iny, sin (27y,) 


where z, and z, are two random numbers from a normal distribution 
N(O, 1). To transform these to two random numbers from a normal 
distribution with mean value of » and standard derivation o, N(u, a), we 
write 


Uy = £19 % pL 
(6.127) 


Un = 20 + B 


where v, and v, are numbers from the normal distribution N (y, o). 

Method 2 is more accurate but it uses more computer time as it 
requires the evaluation of logarithmic and trigonometric functions. The 
first method gives sufficiently accurate results and is commonly used. 

In the previous example of Fig. 6-2, white noise of N(y,o) can be 
introduced, for example, on the values of missile acceleration A, by 
computing v values from the above equations and adding to the values of 
acceleration at each time step. The bias errors (if they exist) are treated 
as constants and are directly added to the values of the variables. 
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Aside from the problems discussed above, in many computer applica- 
tions it is desired to obtain estimates of the mean x and the standard 
deviation s of random parameters. These values are computed using the 
equations 


el 
II 


(6.128) 


where x, is the parameter or the variable and M is the number of samples 
available. The last equation for s is more suitable for digital computer 
calculations since the value of x is not needed for keeping the running 
sums of ae The need for the computation of the mean and standard 
deviation primarily comes in reduction of data and in the case of Monte 
Carlo programs. These are programs with random inputs which are run 
many times to obtain statistical distribution of the parameters of the 
solution. For example, introducing noise with a normal distribution, as 
discussed above, at point c of Fig. 6-2 will result in parameters such as P 
and @ as a function of time with noise superposed. If the statistical 
behavior of these parameters at certain instances of time is required, the 
program should be run many times (say, about 100)* and the mean and 
sigma values calculated at these required times. 


PROBLEMS 


1. Obtain Taylor series expansion of the solution of the equation 


*The number of times a Monte Carlo program is run (or equivalently the number 
of data points) depends on the desired accuracy and confidence intervals of statisti- 
cal parameters (see [9] ). 
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eee eee 


dt 


about the initial condition x = 0 at t = 0. From this series, obtain 
the value of x at t= 0.2. Compare with analytical solution. 
Obtain Taylor series solution of the simultaneous equations 


dz 
= = * > ¥ 
dx 
aa 
dx 


about the initial conditions x = 0,z=1,y = 1. 
Obtain the Taylor series expansion of the equation 


d*y 
dx? 


— —3y2 4 2y 


about the initial conditions x = 1, y = 1, dy/dx = 0. From this 
series, obtain the values of y for x = 1.1, 1.2, and 1.3. 
Obtain the solution of the differential equation 


= 


dx Ay 


with the initial conditions y = 1 for x = 2, for x = 2.1, 2.2, and 2.3, 
using Euler’s method. 
Obtain the solution of the differential equation 


with the initial conditions x = 3, y = 1, for x = 3.05 and 3.10, using 
Euler’s method. 
Obtain the solution of the differential equation 
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10. 


11. 


i. 


3. 
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with the initial conditions y = 1 when x = 1, for x = 1.1 and 1.2. 
Using Milne’s method, formulate the numerical solution of the 
differential equation 


assuming that yo, y,;, ¥5, and y, corresponding to x9, x), X», and 
X, are available. 

Using Milne’s method, formulate the numerical solution of the 
differential equation 


assuming that y,, y,, yo, and y, corresponding to fp, t), tg, and ft. 
are given. 

Derive Eq. (6.51) from Eq. (6.50). 

Formulate the solution of the differential equation 


with the condition x = 0, y = 1, by the Fox-Euler method. 
Point out the difficulty encountered in using the Fox-Euler method 
in the solution of the nonlinear differential equation 


with the initial conditions x = %), y = Yo. 

Formulate the solution of the above problem by Euler’s method of 
starting the solution. 

Formulate the solution of the second-order differential equation 


14. 


8 


16. 


-s. 


18. 


19. 


20. 


pa 


wae 


23. 
24. 
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with the initial conditions x = 0, y = 1 dy/dx = 2, by Milne’s 
predictor-corrector method. 

Formulate the solution of the above equation (Prob. 13) by direct 
substitution of central differences for the derivatives. 

Formulate the solution of the differential equation 


with the initial conditions x = 1, y = 1, by Noumerov’s method. 
Ax4 (4): 5 Z 
Include the term = y, in Eq. (6.90) and obtain the corresponding 


equation to Eq. (6.94) 

Evaluate the solutions given in Table 6-8 for h = 0.01 up to time of 
O43. 

Evaluate the solutions given in Table 6-8 for h = 0.1 up to time of 
1.0. 

Formulate the solution of simultaneous equations of Prob. 2 by the 
Runge-Kutta method. 

Formulate the solution of differential equation of Prob. 13 by the 
Runge-Kutta method. 

Obtain the corresponding equations to Eqs. (6.106) and (6.107) 
using Kutta’s third-order rule of Eq. (6.98). 

Obtain the corresponding equations to Eqs. (6.106) and (6.107) 
using Kutta-Simpson three-eighths rule of Eqs. (6.100). 

Formulate the solution of Prob. 15 by the Runge-Kutta method. 
Write the proportional navigation problem of Fig. 6-2 in terms of 
velocities. In this case Eq. (6.108) becomes 


where V,, and V7 are the velocities of missile and target perpendicular 
toR. In this equation, Vp, = Rw and 


Vu = Vin — Ro 
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26. 
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Differentiating, we get 


ae 
ea eee tat 
dt dt 

Or 


Ay = Ap — Ro - oR 


Carry out the rest of the formulation from Eq. (6.108) using the 
above relation. 

Draw the logic block diagram to represent the nonlinear function y 
as a function of x. 


Prob. 25 


Draw the logic block diagram to represent the nonlinear function y 
as a function of x 


Prob. 26 


Numerical Solution of Ordinary Differential Equations 269 


27. Write the logic of entering normally distributed white noise with a 
u = 0.005 and o = 0.01 ft/sec’ at point c of Fig. 6-2. 
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7 BOUNDARY VALUE 
DIFFERENTIAL 
EQUATIONS AND 
STABILITY OF SOLUTION 
OF DIFFERENTIAL 
EQUATIONS 


BOUNDARY VALUE PROBLEMS 


From the theory of linear differential equations it is known that the 
general solution of an nth-order differential equation will contain n 
arbitrary constants. These constants are evaluated from the given con- 
ditions of the dependent variable and its derivatives at certain values of 
the independent variable. As discussed in the previous chapter, if all of 
these conditions are specified for the value of the independent variable 
xg, Where the solution is desired for x > x,), then the differential 
equation is termed an initial value problem. If some of the conditions of 
the dependent variable are specified at the end point x = L, where the 
solution is desired for x) < x < L, then the problem is termed a two- 
point boundary value problem. In this chapter the numerical solution 
of the boundary value problems will be considered primarily for linear 
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differential equations in order to compare results with analytical solu- 
tions. The methods described can equally well be applied to nonlinear 
differential equations. 

The methods of the previous chapter, together with relations given in 
Chap. 5, can be applied to the solution of boundary value problems. We 
will show that the numerical solutions of the boundary value problems 
can be accomplished either by iterative methods or can be reduced to the 
solution of a set of n simultaneous linear equations in n unknowns. In 
this case the number of equations will depend on the step size (delta) of 
the independent variable. The methods of numerical solution of linear 
simultaneous equations discussed in Chap. 1 can readily be applied to 
evaluate the unknowns of the simultaneous equations. 


Iterative Methods 


Consider the differential equation 


m 
= 2 Fisi y; = ee =) (7.1) 
Xx 


where the superscripts refer to derivatives with respect to x. The above 
differential equation will require m auxiliary conditions for solution. 
Using finite-difference representations, Eq. (7.1) can be put in finite- 
difference form: 


Eso as PS) Fp oe Pp pe (7.2) 


where y,, , is expressed in terms of its values at previous x positions and 
the increment Ax. Since in a boundary value problem the values of y and 
its derivatives at x = x, and x = x, are given, Eq. (7.2) is solved by 
iteration as explained below. 

In order to start Eq. (7.2), several values of y, depending on the 
order of the differential equation (7.1), at x = x9, x = X9 + Ax, x = XQ 
4 2Ax,..., are needed. In boundary value problems not all of these 
values can be obtained from initial conditions given at x = x, since some 
of them should satisfy conditions at the other boundary x = x,. In 
using iteration methods for the solution, all of the values of y for starting 
Eq. (7.2) are either taken from initial conditions at x = x) or are 
assigned trial values. Then Eq. (7.2) is applied successively up to 
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solving for y,, which is the value of y at the boundary x,. If the values 
Of Y,>¥n— 1 Yn -g> ---» match the finite difference representation of the 
boundary condition at x = x, the solution is terminated; if not, the 
trial values of y for starting Eq. (7.2) are altered, usually linearly inter- 
polated as a function of values of y at x, boundary, and Eq. (7.2) is 
reapplied from x = xp. 


Method of Simultaneous Equations 


This method involves the direct substitution of the differential equation 
by an equivalent finite-difference equation and the solution of the 
resulting simultaneous equations. The general equation representing 
Vi, 1 atx;,1 = XQ + 1 Ax (where xp is the boundary and i is the x index) 
is written in terms of previously available y;,y; 4,9; 5,---- The 
boundary conditions at two boundaries, x = xg and x = x,, are also 
formulated in finite-difference form and incorporated with the general 
¥;4 1 equations. The final formulation should result in / equations in / 
unknowns, where 7 is a function of the number of increments of Ax in 
the region x) <x <x, and varies as a function of the order of the 
differential equation and the given boundary conditions. The methods of 
solution described above are used in the following example. 

Example. It is required to find the solution of the differential equation 


2 
oo (7.3) 


with the initial and boundary conditions (¢ = 0, x = 1) and(@t = 1, x = DL). 
The solution of this differential equation satisfying the given boundary 
conditions can be obtained from methods of the Appendix or texts on 
differential equations [1]. This solution is given as 


xX = (2/3) - cosv3 oi 3, + cosV3t + = (7.4) 
sin V3 3 


Equation (7.3) can be replaced by its difference form using central dif- 
ference relations of Table 5.4 of Chapter 5. Since the solution is desired 
for the interval t = O to t = 1, Fig. 7-1(a), this interval is divided into n 
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figure 7-1(a). Increments for the solution of Eq. (7.3). 


equal parts resulting in AT = 1/n. Note that the value of x at t = 0 is 
x), = 1 and at ¢ = 1 is x, = 1. Let us now replace the differential 
equation (7.3) with its central difference equivalent. For t = t, 


nr ee 
_. Se Beak. 
At? ; : 


fee ee 4, - 2) 


ee ee eX 
t+] 1 Gs 3 t. (7.6) 
At? 


n 
a Se (7.7) 


Iterative Method. The general form of Eq. (7.6) can be used to solve 
for X,, 4 as 
ey eg As 
(7.8) 
fot fs et 


where bs Ee . 
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Note that the values of x, and x, are known from the initial (¢ = 0) 
and boundary (t = 1) conditions. The iterative method of solution starts 
by solving Eq. (7.8) fori = 1 


0g ony +e (2) BAP At | (79) 


where t, = At. Since the value of x, = 1 is given in the problem, a value 
for x, should be assumed. Let At = 0.2; this will result in n = 1/0.2 = 5 
increments. Using this value of At and assuming a trial value of x, = 1, 
Eq. (7.9) can be solved for x, and subsequently Eq. (7.8) can be solved 
for x, x4,and x5. This will result in 


XQ = 1 
xy = 1 
x» = 0.888 
(7.10) 
X_ = 0.68544 
x, = 0.424627 
x, = 0.144859 


Since the value of x, = x, from the boundary condition should be 1.0 
(higher than 0.144859 obtained above), let us assume another trial value 
for x, = 2. This results in 


XQ = 1 
X, = 2 
7A 
Xe = 3.21984 
X4 = 3.30930 
Xe = 3.03364 


Now, we passed the boundary value at x,. At this point we can use a 
linear interpolation between values of relations (7.10) and (7.11) to 


OE ————— ————— 
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obtain the next trial value of x,; that is, 


oo 4 
ee 1) - 0.144859) 
= -—;) (7.12) 


= 1.29602 


where the parentheses on the right contain the desired value of x, at the 
boundary and its value from relations (7.10). Using the above value of 
x, and reapplying Eqs. (7.9) and (7.8), we get 


XQ = 1 
x, = 1.29602 
Xq = 1.44452 
(7.13) 
Ke = 1.43568 
i, = 1.27855 
x- = 1.0 


which results in the desired value of boundary condition x,. 

Simultaneous Equations Method. Since the end conditions x, and x, 
are given, x, through x, _ , are the unknowns. Setting the value of x, = 1 
in Eqs. (7.5), writing Eq. (7.6) fori = 2, 3, 4,...,n - 2, and setting the 
value of x, = 1 in Eq. (7.7), we can write n - 1 equations inn - | 
unknowns x, through x, _, as given in Table 7-1. For a given value of 
At the value of n = 1/At can be calculated and the equations of Table 
7-1 can be solved. These equations can be written in matrix notation as 
follows: 


(A)(Q) = (Y) (7.14) 


where (A) is the square matrix of the coefficients of x, through x, _ ,, (Q) 
is the column matrix of x’s, and (Y) is the matrix of the constants given 
on the right side of Table 7-1. From Eq. (7.14), the solution (Q) can be 
obtained by premultiplying both sides of the expression by (A)~! 


(Q) = (A)-!(y) (7.15) 
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The solution was evaluated with a digital computer program to be 
discussed shortly. The results of the solution for various values of Ai 
It is noted that no significant change in the 
solution appears as the value of At is increased from 0.05 to 0.1 to 0.2. 
The analytical solution, evaluated from Eq. (7.4), is also given in Table 


are given in Table 7-2. 


7-2. 
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Computer Program for the Solution of Eq. (7.3) 


The following computer program has been written in BASIC program- 
ming language [2] for the numerical evaluation of the above solution. 
In the program, matrix methods are used to solve the simultaneous 
In the BASIC programming language rows 
and columns of matrices are started from zero; thus the first element of 


equations given in Table 7-1. 


the matrix is the (0, 0) element, and so on. 


Table 7-2. Results of the Solution of Eq. (7.3) 


0.85 
0.90 
0.95 


From 
Eq. (7.4) 


1 
1.08541 
1.16281 
1.23174 
1.29182 
1.34271 
1.38416 
1.41599 
1.32808 
1.45039 
1.45295 
1.44588 
1.42933 
1.40358 
1.36893 
1.32576 
1.27454 
1.21576 
1.14999 
1.07785 
1.0 


1.0 
1.08549 
1.16297 
1.23198 
1.29212 
1.34307 
1.38456 
1.41643 
1.43854 
1.45087 
1.45343 
1.44635 
1.42979 
1.40401 
1.36932 
1.32611 
1.27483 
21598 
1.15014 
E.OFT92 
1.0 


x 


1.0 


1.16338 


129287 


1.38556 


1.43969 


1.45462 


1.43091 


1.37028 


1.27554 


1.15053 


1.0 


At=0.2 
1.0 


1.29602 


1.44453 


1.43568 


1.27856 


1.0 
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10 LET T=0.1 

20 LET N=1/T 

30 LET M=N-2 

32. DIMA (25,25),Y(25,0),B(25,25) ,0(25,0) 
35 LET Z=-2+3*T*T 

40 MAT A=ZER(M,M) 

45 MAT Y=ZER(M,0) 

50 LET A(0,0)=Z 

55 LET A(0,1)=1 

60 LET Y(0,0)=Tt3-1 

65 FOR J=1TOM-1 

70. LET A(J,J-1)=1 

75 LET A(J,J)=Z 

80 LET A(J,J+1)=1 

85 LET Y(J,0)=(J+1)*T*(Tt2) 
90 NEXTJ 

95 LET A(M,M-1)=1 

100 LET A(M,M)=Z 

105 LET Y(M,0)=(M+1)*T*(T*2)-1 
106 MAT PRINT A: 

107. PRINT 

108 MAT PRINT Y 

110 MAT B=ZER(M,M) 

115 MAT B=INV(A) 

120 MAT Q=ZER(M,0) 

125 MAT Q=B*Y 

127 PRINT’“ANSWER MATRIX OQ” 
128 PRINT 

130 MAT PRINT O 

140 LET K=31.5 

145 FOR D=0TO1STEP T 

150 LET X=(((2/3)-COS(K))/SIN(K)) *SIN(K*D)+COS(K*D)+D/3 
155 PRINT “D= “D,“X= “xX 
156 PRINT 

160 NEXTD 

170 END 


Statement 10 of the program specifies the value of At = T = 0.1. 
Statement 20 computes the number of increments N = 1/T= 10. Since 
Table 7-1 has N — 1 equations, the number of equations will be 9. As 
mentioned previously, the matrix notation of BASIC language starts 
from the zeroth row; this will mean that equations are numbered from 
zero to 8. Or, by statement 30,M = N - 2=8. Statement 32 reserves 
computer memory space for matrices used in the program. Statement 


Boundary Value and Stability of Solution of Differential Equations 279 


35 computes the diagonal coefficients of x,, x5, X3,.-- of Table 7-1. 
Statements 40 and 45 set the elements of matrices A and Y of Eq. (7.14) 
equal to zero. Statements 50 through 105 compute the elements of 
matrices A and Y as given in Table 7-1. Statements 106 through 108 
print matrices A and Y. This is done for information only and, of 
course, does not affect succeeding results. The printout of these matrices 
is shown below. 


— 0.999 

0.002 

0.003 

0.004 

Y=} 0.005 
0.006 

0.007 

0.008 

—0.991 


Statement 110 sets elements of matrix B equal to zero. Statement 115 
inverts matrix A and sets this inverse equal to B, that is, B = Aq‘. 
Using this together with Eq. (7.15), the matrix containing the answers 
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will be 
J = BY 


given by statement 125. Statements 127 through 130 print out matrix 
@. The rest of the statements — 140 through 160 — compute and print 
out the analytical solution given by Eq. (7.4). Note that statement 
150 corresponds to algebraic Eq. (7.4). The results of this program are 
shown in Table 7-2. 

From the solutions discussed above it can be seen that the iterative 
solutions are more adaptable to hand calculations and where computer 
routines for the solution of simultaneous equations are not available. 
From Table 7-1 it is seen that the boundary value problem can be re- 
duced to the solution of a set of tridiagonal simultaneous equations. 
The recommended method of solution of this type of system as dis- 
cussed in the next chapter is Thomas’ method. However, if the number 
of equations is small, the use of other methods will not result in 
excessive computer time. 

As was seen in the case of the foregoing example and as will be seen 
in the case of the following example, the iterative method of solution 
requires some judgment in “bracketing’’ the starting values (x, in the 
case of the previous problem). Translating this into a computer program 
may require an excessive number of logical operations. 


Other Types of Boundary Conditions 


In the above example the values of the function x were specified at 
the end points t = 0 and t = 1. In a number of problems the values of 
the derivatives of the function may be specified at the end points 
instead of the values of the function itself. Let us again consider Eq. 
(7.3) as repeated below: 


ea: (7.16) 


but this time with the boundary conditions (t = 0, x = 1) and (¢ = 1, 
dx/dt = 1). Note that in this case the derivative at t = 1 is specified. We 
again proceed with the formulation given in Fig. 7-1(a) and Eqs. (7.5), 
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(7.6), and (7.7), but introduce the derivative at ¢, = 1. Consider Fig. 
7-1(b), which is the end boundary of the problem. In this case the 
fictitious point t, , , is added. The value of the derivative at t, can be 
written using central differences 


ai Sy 
(2) - (7.17) 
t 


Xn-2 Xn-] Xn Xn+] 
—e 
tn-2 th-1 th tht] 


figure 7-1(b). Boundary value att, = 1. 


iterative Method. The solution of the problem in this case will pri- 
marily involve the application of Eq. (7.8) for i = 1 through n instead 
of n — 1 as was the case in the previous example. This allows the calcu- 
lation of x, , ,, which is required in the derivative Eq. (7.17). Note that 
from the boundary condition the final value of the derivative should be 
unity 


(7.18) 


Assuming a trial value for x, = —- 1 and using At = 0.2 as was done 
previously, Eq. (7.8) will result in 
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ee! 

x, = -l 

Xo = —2.872 

x, = —4.38336 = 
x, = 5.34472 

fe = 5.68971 

Xg = ~5.20477 

D = 0.349858 


Since this value is short of the boundary condition at x,, we assume 


another trial value of x; = - 2. This results in 
%, = 1 
%, = —2 
Xo = —4.752 
%_ = -6.91776 Am 
x, = —8.22939 
xX, = —8.52149 
X%_ = —7.75101 
D = 1.19594 


Using relations (7.19) and (7.20) and interpolating for the trial value of 
x, for the next iteration, we get 


2 
font 
| 


1.19594 — 0.349858 
= —1.76841 


oS, at" ~ 0.349858) (7.21) 


Using this value for the next trial, we get 
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= 1 

x, = -1.76841 

Xo = —4.31662 

X_. = —6.33083 (7.22) 
xX, = —7.56134 

x, = —7.85249 

Xe = —7.16134 

D> = 07999997 


This solution satisfies the boundary and initial conditions. 
Simultaneous Equations Method. Equating the slope from Eq. (7.17) 
to the given boundary condition (dx/di) = 1 at t = t, = 1, we get 


Mee scoNi-s «_, (7.23) 


The set of difference equations for this problem will be identical to the 
set given in Table 7-1 except for the last equation of the set and an 
additional equation for - The last equation of Table 7-1 (for t, _,) 
becomes 


eee 2 + BA) + x = t,_, (AD? (7.24) 


Note that in the above equation x, is an unknown. The equation for ¢, 
can be written 


Btrare Aa? 4 SAL) og ety (AD? (7.25) 


In this expression, the value of x, , , can be substituted from Eq. (7.23), 
resulting in 


ee 2s BM) = ¢ AN? ~ SAE (7.26) 


Thus, the substitution of Eq. (7.24) in place of the equation for Ls 
of Table 7-1 and the addition of Eq. (7.26) for t, will incorporate the 
new boundary condition in the problem. This will result in n equations 
in n unknowns, x, through x,. 
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Use of Forward and Backward Differences in 
Incorporating Boundary Conditions 


The boundary conditions can also be expressed in terms of forward 
and backward differences. Consider the second-order differential equation 


Z 
y” = ZS tes Ve: y> y’) (7.27) 


dx? 


with the boundary conditions (x = ka, Gy/dx = yp) ANGIX = x,y = yy), 
Denoting the values of y at discrete x intervals as y, and considering 
Fig. 7-2, Eq. (7.27) can be written in. central difference form for point 


At) Y1 ¥2 Yi-1 Yi Yit+l Yn-2 Yn-1 Yn 
XO x] x2 Xj-] Xj Xj+] Xn-2 Xn-] Xn 


figure 7-2. Use of forward and backward differences in boundary value problems. 


i. Using difference expressions from Table 5-4 of Chapter 5 for y” and y’, 


V4, - 29. 4+ ¥2_ re i sae 
1 1 t=4 4 i(s Yo rs 1 ) (7.28) 
Ax? YAMS 


Transposing the unknown y’s to one side of the equation, we get 


8:9; ,1 Vp Vi-p AD = w; (7.29) 


L 


where g, is a function of the given variables and w, is a function of known 
values of x,. The above expression can be written fori = 1 toi =n-1 
corresponding to values of y for x, through x, 


81 (Vo Y> Yor Ax) = Wy 


85 (Va, o> Vy Ax) = Wo 
at ie OR STOR : (7.30) 


En -2.Vn- 1) Yn-2 Yn- 3 AX) = Wy 9 


Bn -1 Yn 1 Yn—g2 AX) = Way 
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The above relations are (n — 1) equationsin n+ 1 unknownsyp, y,, .--, y,- 
The solution can be obtained only if two additional equations are given. 
These additional equations are derived from the boundary conditions. 

The expression for the derivative of the function at yg in terms of the 
forward differences of Table 5-3 of Chapter 5 results in 


dy 


1 
wid = Dy, = ——(-38y, + 4y, - yo) 71.31 
= % Ee. ihe ; : 2 


Equating this to the given value of the derivative at the boundary 
x = 0, we get 


—3¥q + 4y, - Yo = 2fAx) 6 | (1.32) 


The expression for the derivative of the function at y, in terms of the 
backward differences of Table 5-2 of Chapter 5 results in 


dy l 
> oe ——_ (3y, — 4), 1 #9 2) (7.33) 
2Ax 


dx 
= =Xpn 


Equating this to the given boundary condition at x = x_, we get 
ees ae 2(Ax) y* (7.34) 


Equations (7.32) and (7.34) along with the set of equations (7.30) 
constitute (n + 1) equations to be solved for (n + 1) unknowns yo 
through y,. Note that in the solution of Eq. (7.27) the differentials 
were always replaced by differences with errors of order h? 

Example. Consider the solution of the differential equation 


ey 4 ¥- 4° x" (7.35354 


where primes denote derivatives with respect to x, subject to the initial 
and boundary conditions (x = QO, dy/dx = 1) and (x = 1, dy/dx = OQ). 
Replacing the differential equation with central differences of e = O(h7) 
from Table 5-4 of Chapter 5, we get 
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Us eto ek widest Sa age 5 eae (7.36) 
Ax2 2Ax 


Transposing the unknown y’s to one side, we get 
¥,, 402 - Ax) + y,(-4 - 2Ax*) + y,.1(2 + Ax) = x" GAx*) (7.37) 


Using Eq. (7.32) for the given derivative (dy/dx = 1) at x = 0, we get 
—3y9 + 49, - Yo = 2(Ax) (7.38) 

Using Eq. (7.34) for the given derivative (dy/dx = 0) at x = 1, we get 
ee ee (7.39) 


Combining Eq. (7.38), Eq. (7.37) fori = 1, 2,...,(n — 1) and Eq. (7.39), 
we get the set of equations 


— 39 + Ay, - Yo = 2Ax 
(2+ Axdyy + (-4 — 2Ax*)y, + (2 - Axdyy = 2(Ax)? x? 


(2+ Andy, +4 - 2Ax2) y, + (2 - Ax)yg = AE we (7.40) 


ee 68 2 Se OS 0 ee 8S 8 Oe 6 Oe eS OP SE, Se 1 8S SS ee 6 ee 2 ee a ee ae 8 8 


a+ Ax)y, 9 + (-4- 7 isc al + (2 - Ax)y, = x2 


EE es Weg 4 + 3Yn = 0 


Equations (7.40) constitute a set of (n + 1) equationsin (n + 1) unknown 
yg through y,. In the above equations, x; = i Ax. Equations (7.40) are 
solved for the unknowns for Ax = 0.05, Ax = 0.1, and Ax = 0.2. The 
results are given in Table 7-3. 


Solution of Simultaneous Differential 
Equations 


As mentioned in the previous chapter, a differential equation of nth 
order can be reduced to n first-order differential equations. Thus, a set 
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Table 7-3. Solution of Eq. (7.35) 


= y 
Ax =0.10 Ax =0.20 


0 -2.11359 -2.11158 -2.10602 
0.05 -2.06499 

0.10 | -2.01918 -2.01718 

0.15 -1.97618 

0.20 -1.93598 -1.92297 -1.92823 
0.25 | -1.89858 

0.30 -1.86397 -1.86193 

0.35 -1.83214 

0.40 -1.80305 -1.80096 -1.79485 
0.45 -1.77669 

0.50 -1.75301 -1.75085 

0.55 -1.73198 

0.60 x7 £302 -1.71126 -1.70448 
0.65 -1.69760 

0.70 -1.68412 -1.68173 

0.75 -1.67302 

0.80 -1.66419 -1.66163 -1.65379 
0.85 -1.65754 

0.90 -1.65294 -1.65018 

0.95 -1.65027 


-1.64938 -1.64636 -1.63690 


of differential equations containing higher-order derivatives can be re- 
duced to a set containing only first-order derivatives. The solution of 
sets of first-order boundary value differential equations is obtained using 
the procedures discussed above. 

Consider the set of first-order differential equations 


a 
| 


* = if(x, 9,0 
(7.41 
= g(x,y, 


be 4 
| 


where primes denote derivatives with respect to ¢, with the initial and 
boundary conditions (t = 0,x = x,) and (¢=1,y=y,). The above 
expressions can be written in difference form using forward differences 
of order h from Table 5-3 of Chapter 5. This results in 
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Xe417- % 
= ; = f (x., Se t) 
(7.42) 
a ee eat 
— 5 = g(x, ee t.) 
Collecting the unknowns x and y to one side, we get 
AAX, 4) Xp Vp At) = Ww, (7.43) 


| 
N 


f(s Vina Vp AD = 2; 


where h and f are functions of the given variables and w and 2z are 
functions of the independent variables t and At. Writing these equations 
fori = 0 toi = n —1 will result in 2n equations and 2(n + 1) unknowns, 
that is, x) through x, and yg through y,. Since the values of x, and y,, 
are given, these can be substituted into the equations, thereby reducing 
the number of unknowns to 2n. At this point, the set of simultaneous 
equations can be solved by methods discussed in Chap. 1. 

Example. Consider the second-order differential equation given in 
Eq. (7.35) and repeated below: 


yy = yt y+ x? (7.44) 


with the boundary conditions (x = 0, y’= 1) and (x = 1, y’= 0). Denoting 
y’ = z, Eq. (7.44) can be reduced to two first-order differential equations 


yo = 2 
(7.45) 


oe Saye x" 


In the case of these equations, the corresponding equations to Eq. (7.42) 
become 


oS hee 
Ax : 
(7.46) 
ae ee 9 
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This results: 


a her Ee (7.47) 
2,1 + 2,(-1 - Ax) - Axy,; = Ax (x)? 


Dividing the interval 0 < x < 1 into n equal increments Ax, Eqs. (7.47) 
can be written for i = 0 toi = n —1 for the unknowns Yq to ¥, and 2) 
to z,, 


ag: +7, = Ps 
V4 +¥o -Axz, = 0 
=I #¥5 —~Axz9 — 0 
= oe eS -Axz, 44 = 0 
—Axy +2, = (1+ Ax)z) + (Ax) x9 
—Axy, -(1+ Ax)z, +2, ~ Ai 
-Axyo 
—ARY 4 -(1+4+ Ax)z,_1 = Agte, 7° -Z, 


(7.48) 


Setting the values z) = 1 and z, = 0 in the above equations from the 
given boundary conditions, we will obtain 2” equations in 2n unknowns 
yg through y, and z, through z, _,. 

The solutions of these equations are obtained for Ax = 0.1 and Ax = 
0.2 corresponding to n = 1/0.1 = 10 and n= 1/0.2 = 5 intervals between 
x = 0 tox = 1.0. These results are presented in Table 7-4, and can be 
compared to the solution given in Table 7-3. Note that the solution of 
Eqs. (7.48) also gives the values of z = dy/dx in addition to the values 
of y. The difference between values of y between Tables 7-3 and 7-4 
comes primarily from the fact that the formulation of Eq. (7.46) con- 
tains an error of order h while formulation of Eq. (7.36) contains an 
error of order h?. 
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Table 7-4. Solution of Eqs. (7.48) 


0 -2.11814 1.0* -2.11859 LoU* 

0.1 _ -2.01814 0.888186 

0.2 1 FZ9SZ 0.776191 -1.91859 0.776283 
0.3 -1.85170 0.664878 

0.4 -1.78521 0555195 -1.76333 0.555822 
0.5 -1.72969 0.448193 

0.6 -1.68488 0.345043 -1.65217 0.346320 
0.7 -1.65037 0.247060 

0.8 -1.62566 0.153729 -1.58290 0.157150 
0.9 -1.61009 0.072735 


-1.60282 0.0* -1.55147 0.0* 


*Inserted from boundary conditions. 


STABILITY OF SOLUTION OF 
DIFFERENTIAL EQUATION 


As discussed here and in the previous chapter, the numerical solutions 
of differential equations depend on the particular finite-difference repre- 
sentation of the equation and the increment of the independent variable, 
Ax or At. Correlating the stability and the accuracy of the solution of 
the difference representation with the size of the increment is an import- 
ant step toward the justification of the finite-difference representation 
(see stability discussion of the previous chapter). Unfortunately, this 
correlation can be obtained analytically in a limited number of cases. In 
many practical applications the correlation of Ax and At with the 
accuracy of solution is made by running particular finite difference for- 
mulations on the computer and examining the resulting solutions. 

Analytic methods of stability and accuracy determination of finite- 
difference formulations involve the use of difference-differential equa- 
tions, as described below. 


ANALYTIC SOLUTION OF DIFFERENCE- 
DIFFERENTIAL EQUATIONS 


The finite-difference formulations of differential equations can be 
represented in the form 
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oti FT) + v,x{t+ @- IT] + VoXxlt ae See i 
(7.49) 


Same ep v x(t) = f (t) 


where T is the increment in ¢ and 7 is an integer. Using the previously 
given notation 


Eix(t) = x(t + iT) (7.50) 


Eq. (7.49) becomes 
(Ei + vy Bi) + vy BF +. + v,)xlt) = fle (7.51) 


Following [3], the complementary solution of the above equation x, 
can be obtained by solving 


(Ei + v, Bio} + vont? 4 ose + vx, (0) = 0 (7323 


The solution of Eq. (7.52) is obtained by assuming 
2 i = Cem : (7-53) 


where C is an arbitrary constant, and m is a number to be determined. 
Substituting Eq. (7.53) in Eq. (7.52), we have 


Cemt(emiT Rr cee a yy emi- BT oS ade 0 (7.54) 


Dividing through by Ce™! and substituting q = e”", we get 


So a2 ee ee (755) 


l 


(Note that this equation is identical to Eq. (7.52) with q’s replacing E’s.) 
Assuming that t = nT, where n is an integer, the solution (7.53) becomes 


wm, (nT) = Ce™™! = Cg” (7.56) 
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— The roots of Eq. (7.55) will give the values of g to be used in the solution 
equation (7.56). The roots of Eq. (7.55) can be obtained by standard 
polynomial root finding methods discussed in Chap. 3. Three cases are 
encountered after the roots are obtained. They are: 
1. Distinct Roots. In which case the solution (7.56) becomes 


x,(nT) = Cia)" + Codg” + Caga” + --- + C.q,” wae ee 


tt 


where 9, 95, ---» 9; represent the roots of the polynomial (7.55). 

2. Complex Roots. In this case the roots will be complex conjugate 
as the coefficients of Eq. (7.55) are real (see Chap. 3). The conjugate 
roots q,, @5 will be in the form 


ge + OF Gs = eS (7.58) 
where a and b are constants and j* = —1. These roots can be written as 
G = Re!” do = Re~!° Ch599 


where R = va” + b* and @ = tan~=!(b/a). The corresponding solution 
may be expressed 


x,(nT) = Cyq1" + CoQ5” 
(7.60) 


= R"(C, ein? + ee) 


Substituting etin® _ cogné + j sinn@ in the above equation, the solution 
becomes 
x. (nT) = Bee cosnd + Cy sin n@) (7.61) 


where C; and Cy, are new constants. 
3. Repeated Roots. For repeated roots q, = qd, = q the solution 
is expressed as 


Ain gC, + Cyn) (7.62) 
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The particular solution of Eq. (7.49), X,, can be obtained in analogy 
with the differential equation theory [1]. In obtaining particular solu- 
tions we assume several types of solution, depending on the form of the 
function f(t), in Eq. (7.51). This solution is then substituted in Eq. 
(7.51) and the constants are evaluated. The following types of f(# are 
often encountered in practice. 

1. f(t) = A®t = APnT where A and k are constants. In this case a 
particular solution of the type 


x aT? = CARnT (7.63) 


is assumed and this solution is substituted in Eq. (7.51). The resulting 
equation is then solved for the constant C. Thus, the complete solution 
of the differential equation becomes 


X = x (nT) 4 x, (nT) (7.64) 


2. f(t) = At™ = A(nT)™ where A is a constant and m is an integer. 
In this case, a solution of the type 


x, (nT) = C, (nT) # Care * re oe (7.65) 


mt+l 
is assumed and this solution is substituted in the difference equation 
(7.51). The values of C,, C,, ..-,C,,, 1 are computed by equating the 
coefficients of equal powers of n on both sides of the resulting equation. 

3. f(t) = snmAT or cosAT = sinAnT or cosAnT. In this case the 
exponential equivalent of these functions can be written and the solution 
can be obtained by the method of case 1 above. Alternatively, a solution 
of the type 


x (nT) = C, sinAnT + C, cos AnT (7.66) 


can be assumed and substituted in Eq. (7.51). The values of the con- 
stants C, and C, are obtained by equating the coefficients of sin AnT and 
cos AnT terms on both sides of the equation, respectively. 

From the solutions discussed above it can be seen that the comple- 
mentary solution (7.56) represents the transient part of the solution as 
q’s are raised to the number of the time step n. Thus, since q’s in finite 
difference formulations are usually functions of the time increment T, 
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the complementary solution may be unstable for some region of T values. 
On the other hand, the particular solution follows the “driving input” of 
the difference equation as seen from the three cases above. For this 
reason the stability of a specific finite difference solution may be 
determined from an examination of the complementary solution of its 
difference-differential equation. The regions of T values for a stable 
solution are chosen by an examination of the complementary solution of 
the difference equation. 

Example. Consider the error analysis of the finite-difference solution 
of the differential equation 


ooo a (7.67) 


with the initial condition t = 0, y = 1. 

The analytic solution of the equation can be obtained by standard 
methods of solution of differential equations. See Appendix on differen- 
tial equations. The complementary solution is the solution satisfying 


dy + 2y = 0 

dt 
or 

(D+ 2)y = 0 (7.68) 
where D is the derivative operator. The root of this equation is D = — 2 


and the complementary solution becomes 
y = Ce~*t (7.69) 
The particular solution is obtained by assuming a solution of the type 
Y, = At+B (7.70) 
and substituting in Eq. (7.67) 


A+ 2At+ 2B = 3t (7.2%) 
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Equating like powers of t, we have 
ae ae 
A = : B = 


The total analytic solution of the differential equation (7.67) becomes 
et 5g oy 8 
y = Cet, 3; 3 (7.72) 


Setting the initial condition t = 0, y = 1 in Eq. (7.72), we obtain C = 7/4. 
Or Eq. (7.72) with this value of C becomes 


ey Pe are 3 
y= ig + * aie (dii2) 


which is the complete analytic solution of differential equation (7.67). 
Equation (7.67) can be represented in finite-difference form as 


y or 
oe Sy = Bi. = nT (7.74) 
where T is the increment in t, that is, T = At. Denoting 2T - 1 =a, 


Eq. (7.74) becomes 


(E+ ay, = 3(nT)T (7.75) 


The complementary solution is obtained by setting the right side of this 
equation equal to zero: 


(E+ ay, = 0 (7.76) 


The root of this equation is g = —a, and the complementary solution 
becomes 


Oneom = EF a" (7.77) 
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From the above complementary solution it can be seen that for large 
n and a2 | the solution will grow beyond bounds. Therefore, the 
complementary solution will be stable only for 


a= 21 —-1<') 
or 


eA (7.78) 


The particular solution of Eq. (7.75) is obtained by assuming a particular 
solution of the type 


Onan = A(nT) + B (7.79) 


and substituting in Eq. (7.75). This will result in 
Ag + 1)T + Ba oAG@Tl) + ab =. 387) T (7.80) 


Equating the coefficients of like powers of n on both sides of the 
equation, we get 


3T 


7 es ae 
oe - 
: (7.81) 
ite Bors az 
2 : 


In the above, the substitution 1 + a = 2T is used. Thus, the complete 
finite-difference solution of difference equation (7.74) will be the sum 
of Eqs. (7.77) and (7.79) with the constants of Eq. (7.81) 


z = n,3 i 
t= ytnl) = €C a)" + 5 (nT) 7 (7.82) 


The constant C is evaluated by using the initial condition t = nT = 0, 
y =0, orn =0, y=0. Using this in Eq. (7.82) will result in C = 7/4, or 


y(nT) = £(-a)" -- 5 (nT) =: : (7.83) 
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Note that -a = 1 -2T. Using this relation in the above, and subtracting 
Eq. (7.83) from analytic solution (7.73), we get (set t = nT in Eq. (7.73)): 


e = y(t) — y(nT) = ze = Mes ge (7.84) 


where « is the error between the analytic and finite-difference solutions. 
Note that the particular solution does not affect the error term as it is 
canceled by the corresponding analytic solution. The exponential 
function in (7.84) can be expanded in Taylor series as follows: 


zs 2 3 n 
= ( _ OT 4 = = = : ) (7.85) 


Thus, the effective error of Eq. (7.84) is introduced by neglecting the 
second and higher powers of T in the difference solution represented by 
(1 —2T)”. The values of percentage error («/y ) (100) (using Eqs. (7.73) 
and (7.74)) were evaluated for various values of ¢ using T as a parameter. 
A plot of this is given in Fig. 7-3. 

Example. As an example of the above analysis for a second-order 
differential equation, consider the differential equation 


v4 
Sue ee (7.86) 
dt? 


with the initial conditions t = 0, x = 1, and dx/dt = 0. The analytic 
solution of Eq. (7.86) is of the form 


x = Asinwt + BcoswT (7.87) 


where A and B are constants. The initial conditions will establish the 
values of A and B as follows 


A= 0 B= 1 
Using these, solution (7.87) becomes 


x(t) = x(nT) = coswt = cosn(wT) (7.88) 
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figure 7-3. Error in difference formulation and 
the solution of Eq. (7.67) vs. t. 


where t = nT (nis an integer, and T is the time step) is substituted for 
calculations to be performed later. 

The difference representation of Eq. (7.86) using central differences of 
order T2 can be written 


oc oes 2x, Ay 


= ors So (7.89) 


Shifting the index n by one unit, the above equation can be written 
a ee = 0 (7.90) 


In analogy to Eq. (7.52), the above becomes 
[E? + (w*T? ~ 2)E + Ix, = 0 (7.91) 
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The roots of this equation are 


Ege OF tT? 2 oF 4 
ee sh 5 eet Me (7.92) 


Considering the part under the radical, three cases will arise 
Case 1. (2 T? — 2)? > 4 or wT > 2 the roots g, and qo will be real 
valued and distinct. The solution will be of the type given by Eq. (7.57): 


x(nT) = C,q," + Cogn" | (7.93) 


This is clearly a nonoscillatory solution and is unacceptable since the 
analytic solution (7.88) is oscillatory. 

Case 2. (a? T? - 2) = 4 or wT = 2 double root g; = qo = 4. The 
solution will be of the type given by Eq. (7.62): 


x(nT) = q”"(C, + Cy n) (7.94) 
This is again clearly a nonoscillatory solution, and is unacceptable for the 
same reason as in case 1. 


Case 3. ez T? — 2) < 4 or wT < 2 complex conjugate roots of the 
type given by Eq. (7.58) 


= (7.95) 
9g =a- 5); ; 


with a = —(@2T2 — 2)/2 and b =(1/2)/4 —- @2T? - 2)’. The solution 


will be of the type given by Eq. (7.61) 
x(nT) = R"(C, cosné + Cy sinné) (7.96) 


with 


(7.97) 
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The constants Cy and ca can be obtained using the initial conditions for 
t= nT =Oorn=0. For n = 0 and x = 1, Eq. (7.96) will give C, = 1. 
The initial condition on the derivative dx/dt at t= O can be incorporated 
by writing the difference equation for the derivative as 


xin + DT) - x{@- DT _ 9 (7.98) 
25 


OT 
x[(n + 1)T] = x[(n —- 1)T] at n = O 


or using Eq. (7.96) with R = 1, 


oF cos(n + 144+ Cy sin(n + 1)6 
(7.99) 


= Ci cosa (n= 10 + C5 sin(n — Ll)6@ 


Setting n = 0 and equating the coefficients of sine and cosine terms, 
respectively, we get 


C, = 0 (7.100) 


2 
Thus the solution of difference equation (7.90) with the appropriate 
initial conditions becomes 


x(nT) = cosné (7.101) 


where 6 is given by Eq. (7.97). 

Comparisons of solutions (7.101) and (7.88) will show that the value 
of T should be so chosen as to make 0 ~ wT. Note that w is given in the 
problem equation (7.86). Using Eq. (7.97) together with the values of a 
and b in terms of wT, several values of 6 as a function of wT were calcu- 
lated as given in the following table. From this table it is noted that 
values of T corresponding to wT of up to 0.2 seem acceptable. 
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oT 0 
0 0 
| .100042 
2 200335 
2 201137 
4 .402716 
2 50536 
6 .609385 
Ef 3 715142 
8 823034 
— BA5531 
:: 1.0472 
se 1.16473 
1.2 r28/ 
PROBLEMS 

1. Formulate the solution of the differential equation 
d*x 5%, 3 2 mg 
dt? dt 
subject to the initial and boundary conditions (t = 0, x = 1) and 
(t = 2, dx/dt = 3), by using central differences of order T? and 
dividing the interval of 0 < ¢ < 2 into n equal parts. 

2. Formulate Prob. 1 with the boundary conditions (t = 0, dx/dt = 0) 
and(t = 2, dx/dt = 1). 

3. Formulate Prob. 2 using forward and backward differences to rep- 
resent the boundary conditions. 

4. Formulate the solution of the differential equation 


subject to the condition y = y=Qwhenx=1forO<x <1. 
Formulate the solution of the differential equation 


302 


10. 
iL. 


12. 


13. 
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subject to the condition y = 1, dy/dx = 0 when x= 1 forO < x < 1. 
Obtain the solution of the boundary value differential equation 


2 
a2 gf 2 6 = 0 


dx2 dx 


with y = 1 at x =O and y = 2 at x = 1, by the iteration method. 
Obtain the solution of the differential equation 


with dy/dx = y =O at x = 1, by the iteration method. 

Solve Prob. 7 for boundary condition y = 1 at x=Oand y= 2 at 
x= 1. 

Obtain the solution of Eq. (7.16) by the simultaneous equations 
method and compare with results given by relations (7.22). 

Obtain the solutions given for Ax = 0.2 of Table 7-3. 

Formulate the solution of the sets of differential equations 


ee 
dt t t 
es eae 
dt t t 


subject to the conditions x = 0 when t = 0, y = 1 whent = 1. 
Solve the difference-differential equation 


(E* _ 5E + 6)y = ef 


Solve the difference-differential equation 


14. 


15. 


16. 


it. 
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fe iy tS 


subject to the conditions y = 0 when t = 0 and y = 1 when = 1. 
Solve the difference-differential equation 


(E2 - 2EF + 4)y = x? 


subject to the conditions y = 0 when x = 0 and dy/dx = 1 when 
x =0. 

Discuss the stability of the solution of the central difference for- 
mulation of 


by examining the complementary solution of the difference-differ- 
ential equation. 

Discuss the stability of solution of the central difference representa- 
tion of the equation 


using the complementary solution of difference-differential equation. 
Discuss the accumulation of error in the central difference formu- 
lation of the differential equation 


subject to the conditions y = 0, dy/dx = 1 when x = 0. The analytical 
solution of the above equation is 


1 


=a > 
ea te + C,x+ Cy, 
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18. 


19. 
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Discuss the accumulation of error in the forward difference formu- 
lation of the differential equation 


dx 
dt 


subject to the condition x = 5 when t = 0. The analytical solution 
of the problem is 


ce Oe 


Discuss the accumulation of error in the forward difference formu- 
lation of the differential equation 


d 
Se ye 


dx 


subject to the conditions y = 1 when x = 0. The analytical solution 
of the problem is 


ge 1x + Ce" 
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8 NUMERICAL 
SOLUTION OF PARTIAL 
DIFFERENTIAL 
EQUATIONS 


The numerical solution of partial differential equations can be a 
subject of extensive study by itself. There are several excellent texts 
and references written on this subject, including [1, 2]. The presentation 
of the subject given in the following pages should be considered only as 
an introduction rather than as a complete treatment. 

The discussions start by giving simple examples to illustrate the general 
concepts. The generalized forms of the solutions and addition methods 
of solutions are given after the introductory material. 

A second-order partial differential equation can be written in the form 


2 2 2 | 
te ye" + B(x, y) ow + i oa a ee = 0 
x? dxdy ay2 ax dy 
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where A, B, C, and f are functions of the given variables. If f is a linear 
function of the variables in parantheses, the partial differential equation 
(8.1) is considered to be a /inear partial differential equation. If f is a 
nonlinear function of the given variables, the partial differential equation 
(8.1) is considered to be a quasi-linear equation since the second-order 
derivatives 0*u/dx*, 0*u/dxdy, ..., appear in the first power only. The 
partial differential equation (8.1) is categorized depending on the rela- 
tionship of A, B, and C coefficients as follows: 


B2 —~ 4AC < 0 © elliptic equation 
B2 — 4AC = 0 _ parabolic equation 


B2 — 4AC > 0 _ hyperbolic equation 


Examples of the above categories, often encountered in practice, are 
as follows: 


2 2 
Laplace’s equation a + iat 0 elliptic 
Ax Oy 
; A u Ou : 
Heat equation ite = Eee parabolic 
Ax? at 


(where a is a constant and x and ¢ are space and time coordinates). 


2 2 
ce eee hyperbolic 


Wave equation it 
ox? ot? 


(where c is a constant and x and ¢ are space and time coordinates). 

The above partial differential equations are written in the cartesian 
(x,y) coordinate system. These equations can also be written in other 
coordinate systems such as cylindrical and spherical. The following 
discussion, however, will be limited to cartesian coordinate system 
representations. 
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PARTIAL DIFFERENCE OPERATORS IN 
CARTESIAN COORDINATE SYSTEM 


Consider the function 


z = f(x,y) (8.2) 


where x,y are the cartesian coordinate system (Fig. 8-1), and the partial 
derivatives are denoted by 


= ve v4 2 
D.. = a = = eo ea e D2x = or z D2 y = d°z 
Ox Oy = dxdy Eee a8 
(8.3) 


Taking the point (0, 0) as the origin the partial derivatives of Eqs. (8.3) 
become (using central differences) 


figure 8-1. Cartesian coordinate system. 
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27 2 

h* D2 = 21,9 - 220,09 + 2-1,0 (8.4) 
3p 3 

ae DS 29 = 82,0. - Maat M16 - 7.2.0 


where A is the increment in the x direction. 
Similarly, denoting k the increment in the y direction, we have 


2k? D,? Zo = 2% 9 - 229 1 + 22% -1 - 20,-2 


The expression for the mixed derivative D = of the function z is 
obtained by applying the derivative operator D, to the existing expression 


for .. Thus 
we Mee 
Dy 29 = D,(D,) = D, Ft) 
= (Z1 4 = Zz, -)) = (z_44 iz Z_4 1) (8.6) 
Akh 


“1. “).-f~ 42 Se 
Akh 


The above expression in analogy to Eqs. (8.4) and (8.5) can be written 


Akh Dy 20 =-24 4 — 8) 4) ea a (8.7) 


SOLUTION OF LAPLACE’S EQUATION 


The Laplace partial differential equation, as previously given, can be 
written in cartesian coordinates as 


2 2 
es (8.8) 
dx? dy? 
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figure 8-2. Solution of Laplace’s equation. 


where V2 is the Laplace operator (not to be confused with backward 
difference operator). Considering Fig. 8-2 and using difference relations 
given by Eqs. (8.4) and (8.5) in Eq. (8.8), we get 


ee +) a aa? oe ee eee 
eee OS (8.9) 
Ax? Ay? 


Setting the values of Ax = Ay = h, Eq. (8.9) becomes 


U, + Ug + Ug + Uy — 4u, = 0 (8.10) 
Using the above relation and assuming the values of u’s as unknowns, 
the solution of Eq. (8.8) can be obtained by solving a series of simul- 
taneous equations as illustrated in the following example: 

Example. Obtain the solution of Laplace’s equation for the geometry 
of Fig. 8-3. In this figure the values of u at the upper and lower boundary 
are set at 1,000, while the values of u at the right and left boundaries 
are set at zero. The values at the corners are arbitrarily set at 500 so as to 
make the discontinuity of approaching the corners from the x and y 
directions equal, i.e., the change from the nearest point to the corner 
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u=500 1,000 1,000 1,000 1,000 1,000 500 


u=500 1,000 1,000 1,000 1,000 1,000 500 


x 


figure 8-3. Example problem. Solution of Laplace’s equation. 


from the x and y directions is 500. The square cross section is divided 
into six equal increments of length h in both x and y directions. This 
will give 25 points or 25 values of u to be calculated. Since the divisions 
of the cross section are symmetrical, only the values of u at the following 
points will be calculated: 


Uy, Ug, Ug, Ug, U7, Ug, Uy4, Uy, and uj, 


Using Eq. (8.10), equations can be written for these unknowns. These 
equations are given in Table 8-1. Note that in obtaining the coefficients 
listed in the table the following relations, arising from the symmetry of 
the problem, were used: 


The solutions of these equations by previously outlined methods of 
Chap. 1 were evaluated using a digital computer. The answers are 
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Table 8-1. Coefficients of Simultaneous Equations 


as follows: 

u, = 900.0 

Uy = 682.692 

u, = 730.769 

Ue = 317.308 

u- = 500.0 (8.11) 
Ug = 907.692 

Uj, = 269.231 

Uyg = 442.308 

Ui, = 500.0 


Note that these values represent the steady-state temperature distribution 
in a homogeneous solid material if u is considered to be the temperature. 


ITERATIVE METHOD OF SOLUTION OF 
LAPLACE’S EQUATION 

The solution of Laplace’s equation (8.8) can be obtained by a simple 
iterative method due to Liebmann. This method is based on the fact 
that the value of u at any point is the average of the four surrounding 
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values as given by Eq. (8.10), that is, 


Uy = f (uy + Ug + Ug + Uy) (8.12) 


The method starts by assuming an initial distribution of u over the 
region and calculating new values at each iteration using Eq. (8.12). The 
procedure is further explained through the following example. 

Example. Again consider Fig. 8-3 of the previous example, but this 
time let us use matrix notation to specify various points of the cross 
section, as given in Fig. 8-4. Using the same conditions at the boundaries 
as given in the previous example, and assuming a value of u = 500 for all 
of the inside points of Fig. 8-4, the matrix values of u at iteration zero 
will be 


6 «+ Column 


0 ] 2 3 ~ 5 
1,000 500 


500 1,000 1,000 1,000 1,000 


NOP WN KH OO me 
= 


500 1,000 1,000 


The first iteration starts by averaging the four values around each of 
the inside points and replacing the old value with this average. The 
averaging starts with element (1, 1), (1, 2),...,(1, 5) and proceeds with 
(2, 1), (2, 2),..., andso on. Thus at the end of the first iteration the 
matrix of u values will be 


ITERATION 1 
500 1000 1000 1000 1000 1000 500 
0 500 625 656.25 664.063 541.016 0 
0 375 500 539.063 550.781 397.949 0 
0 343.75 460.938 500 512.695 352.661 0 
0 335.938 449.219 487.305 500 338.165 0 
0 458.984 602.051 647.339 661.835 500 0 
500 1000 1000 1000 1000 1000 500 


u-—500 


(3, 0) 
. 


(0, 1) 


(1, 1) 


(3, 1) 
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(0, 2) 


1,000 


(0, 3) 
1,000 1,000 
pape ae) 
(2, ») 


1,000 


1,000 


(0, 4) 
1,000 1, ,000 
(1, 4) (1, 5) 
ee 
vee 
eel 


1,000 


(0, 5) 


figure 8-4. Solution of Laplace’s equation by iteration. 


1,000 


(0, 6) 


(2, 6) 


(3, 6) 


(4, 6) 


(5, 6) 


(6, 6) 
500 


Iterations can proceed in a similar manner resulting in 
values of u for the 10th and 20th iterations 


ITERATION 10 
500 1000 
0 500. 
0 317.225 
0 269.077 
0 317.164 
0 499.928 
500 1000 
ITERATION 20 
500 1000 
0 500. 
0 317.308 
0 269.231 
0 317.308 
0 500. 


1000 


1000 
682.775 
500. 
442.202 
499.875 
682.625 
1000 


1000 
682.692 
500. 
442.308 
500. 
682.692 
1000 


1000 
730.923 
557.798 
500. 
557.643 
730.736 
1000 


1000 
730.769 
557.692 
500. 
557.692 
730.769 
1000 


1000 
682.836 
500.125 
442.357 
500. 
682.684 
1000 


1000 
682.692 
500. 
442.308 
500. 
682.692 
1000 


1000 
500.072 
317.375 
269.264 
317.316 
500. 
1000 


1000 
500. 
317.308 
269.231 
317.308 
500. 
1000 
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the following 


500 


oo°o0c Oo 


500 


500 


ooo oO oO 


500 
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Note that the values shown at iteration 20 are completely symmetrical 
and agree with the values of Eq. (8.11) to three significant figures. 

The following digital computer program was written in BASIC pro- 
gramming language [3] to accomplish the iteration procedure discussed 
above. 

Statement 5 of the program reserves space for the matrix u. State- 
ment 10 sets each element of matrix u equal to unity. Statement 15 
multiplies all of the elements of matrix u by 500, which is the initial 
value of u at iteration zero. 


5  DIMU(6,6) 

10 MAT U=CON 

15 MAT U=(500)*U 
25 FOR J=1TO5 
30 LET U(0,J)=1000 
32 LET U(6,J)=1000 
35 NEXTJ 

40 LET U(0,0)=500 
42 LET U(0,6)=500 
44 LET U(6,0)=500 
46 LET U(6,6)=500 
50 LET 2Z=0 

70 FORI=1TO5 
75 LET U(I,0)=0 

80 LET U(I,6)=0 


85 NEXT I 
87 PRINT “ITERATION’Z 
88 PRINT 


90 MAT PRINT U; 

a2. LET Z=Z+1 

95 FOR I=1TO5 

100 FOR J=1TO5 

105 LET U(I,J)=U(I+1,J)+U(1-1,J)+U (1 J+1)+U (1 J-1) 
107. LET U(I,J)=U(I,J)/4 
110 NEXT J 

115 NEXT | 

116 PRINT 

117. PRINT “ITERATION’Z 
120 MAT PRINT U; 

130 IF Z2<10 THEN 92 

135 END 


Statements 25 through 85 enter the values of u at the four boundaries 
of Fig. 8-4. (Note that Z in statement 50 is the number of the iteration.) 
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Statements 87 through 90 print the matrix of u values at zeroth iteration. 
Statements 95 through 115 compute values of u using Eq. (8.12). State- 
ments 116 through 120 print out the answers. Statement 130 sets an 
upper limit to the number of iterations. (Note that statement 130 was 
changed to Z < 20 to obtain iteration 20 given above.) 


GENERAL FORM OF LAPLACE’S EQUATION 


The above formulations are specialized forms of a more general 
relation describing the finite-difference representation of Laplace’s equa- 
tion. In a number of more complicated applications the formulas 
described in the following pages are preferred to the ones given pre- 
viously as they enjoy a faster rate of convergence to a solution. 

The general form of Laplace’s equation can be written in two- 
dimensional form as 


ps 
aS + BR = fly) (8.13) 


where a and £ may be functions of variables x and y, with the condition 
of elliptic equations satisfied. The above equation is known as Poisson’s 
equation and specializes to Laplace’s equation for a = 6 = constant and 
f(x,y) = 0. The finite-difference representation of Eq. (8.13) for the 
equal increments Ax = Ay = hcan be written 


y ux + A, y) + Yo ulx a ee Y3ulx, a ee v4 ulx, fo 


+ ys u(x,y) = g(x,y) (8.14) 


where y’s are functions of x, y, and the increment h. Solving the above 
equation for u(x,y) and denoting the resulting coefficients by v’s, we 
have 


u(x,y) = v,ulx + h, y) + voulx — fh, y) + vgulx, y + h) 
(8.15) 
+ vyzu(x, y — h) + T(x, y) 
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In addition to the previously discussed solutions, several iterative proce- 
dures are available for the solution of the above equation. Assuming that 
the values of the variable u(x, y) are available at iteration n over the x, y 
region and denoting these values by u, (x,y), the following iteration 
procedures can be formulated. 

1. Gauss-Seidel method or the method of successive displacements. 
The iteration formula for this method is 


U4 4'%,¥) = vu x + h, y) + Volt, , 4\% = fh, ¥) + vg, (x, ¥ + h) 


+ vgu, , 4%, y — bh) + 7, (x,y) ao) 


In a given problem the initial values of the function u, (x, y) are given 
along the boundaries, as was the case of Fig. 8-4. Assuming a set of 
values for inside points at iteration zero, Eq. (8.16) can be solved 
successively for points (1, 1), (1, 2), (1, 3),...,(2, 1), ...and so on, at 
each point using new values of u as they become available. After getting 
to point (5, 5) of the figure the second iteration is started from (1, 1). 
Continuing this procedure the final solution of the problem is obtained. 
Note that Eq. (8.16) in the case of Laplace’s equation reduces to the 
previously discussed Liebmann method. 

Equation (8.16) can be slightly modified to improve the rate of 
convergence of the solution. This modification is as follows: 


bg) = o[v,u, (x ree ee ee Vol, , 4x — h, y) + vgu, (x,y + h) 
+ Vygu, , 1%, ¥ — h) + r, (x, y)| — (o — Du, (x, y) 
(8.17) 


The parameter @ is known as the relaxation factor and the methods 
which use this equation are termed as successive-overrelaxation methods. 
Note that for » = 1 Eq. (8.17) reduces to Eq. (8.16). The choice of 
determines the rate of convergence of the solution. No general expression 
for optimum ® (fastest rate of convergence to a solution) is available for 
use in Eq. (8.17). However, in the case of Laplace’s equation 7 = 0 and 
V1 = Vg = Ya = ¥4 = 1/4, Eq. (8.17) reduces to 
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zs 1 : : ae 
- 4p = 045 [en Sen tS Pee (8.18) 


ee Es to 2 Du Gp 


where i and j are indices in the x and y directions. 
The optimum value of for this case is given [1] as 


= 2 
| H (8.19) 
Pei - 7 


where 


= 1 ( 7h “1 
a a ee cos — 


and a and b are the sides of the rectangle. The above expression for 
a = b=1 and small h becomes 


7 h2 
2 


1 cos zh 2 
a ————Es ON 
1 + sinzh Lak 


Although the above value of » applies to the specialized case of Eq. 
(8.18), it is used as a guide in selecting values of in the solution of the 
more general Eq. (8.17). It can be shown with the @ of Eq. (8.20) that 
the successive-overrelaxation method converges 2/7h times faster than 
the Gauss-Seidel Method. 

2.  Alternating-direction implicit method. This method, developed 
by Peaceman and Rachford [1], is along the same lines as the relaxation 
methods discussed above. The application of this method, however, 
involves solution of tridiagonal linear simultaneous equations at each 
stage of iteration. For this reason before discussing the method we will 
describe the solution of this type of simultaneous equations. 


ecosmh ~ 1 — 


i 
II 


(8.20) 


S 
I 
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Consider the set of tridiagonal simultaneous equations 


BG il, sD; 

Zt 

Bptsig Eel Ct gp hy A (8.21) 
AT.) peel ges 2 


where A, B, C, and D’s are constants and T’s are the variables. The solu- 
tion T’s can be expressed 


TT, = 4 
ge ons Gag Tee (8.22) 
Pe fp Oe 
where 

Dy v= And; 4 
Peter? aS 

B, eee 2 oe 

G2 3. 4, ted 2: 

oF oe 

b, = —> = 
l 
Z ee (8.23) 


The above method, which can be derived from gaussian elimination 
method of solving simultaneous equations, is due to Thomas [4]. Tri- 
diagonal simultaneous equations (8.21) arise quite often in numerical 
solutions of partial and ordinary differential equations. 

Getting back to the alternating-direction method of Peaceman and 
Rachford, the basic iteration equations of this method are, for 
Ax = Ay = h, 


Un yg(%y) = u, (x,y) + riu(x + hy) + ae —2h, ¥)-= 2u(x,y)], 4 1/2 


+ rlu(x, y + h) + ulx, y — h) - 2u(x,y)], 


ea = v/g(%y) + rlu(x + kh, y)} + ux — hy, y) - 2u(x, y+ ve 


+ rlu(x, y + h) + ulx, y - A) - 2ulx, y)] 


n+l 


(8.24) 
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where the subscripts refer to iteration number associated with the variable 
u and r isa constant. Assuming that the values of u at iteration nu, (x, y) 
are available, each complete iteration will require the solution of the 
first and then the second of Eqs. (8.24). Mechanization of the first of 
the above equations will result in m tridiagonal simultaneous equations, 
where m is the number of points inside the rectangle. For example, in 
the case of the problem of Fig. 8-4, the first of Eqs. (8.24) will give 


(1,1) = u(1,)) + rfu(,2) + uM,0) - 2uQ1, D1. 12 
+ rlu(2,)) + u(0,1) — 2u0,D]_, 


u, + 1/2 


(1,2) = u,(,2) + rlw(,3) + wD —- 2u(1,2)] 
+ r{u(2,2) + u(0,2) - 2u(1,2)], 


nt 1/2 (8.25) 


4 1/2 (2,9) = u, (5,5) + r[w(5,6) + u(5,4) — 2u(5,9)),, V2 
= pryco.)) + u4,5)>— 2u(5,5)] 


Transferring the unknowns of the above equations to one side, we get 


eet Bon (1.2) + Con, (1.8) = Do (8.26) 


Aor . 1 (94) rs Bos le: 1 (9,9) = Dos 


The solutions of the above equations by the procedure outlined in 
Eqs. (8.22) and (8.23) will result in values of u at iteration n + 1/2. In 
Eqs. (8.26), A, B, C, and D are known quantities which are functions of 
previously available u, (x,y) After calculating the values of u, ,, yo, the 
second set of Eqs. (8.24) are used in much the same manner to compute 
n+ 1: Note that the first set of Eqs. (8.24) represents a row iteration 
(x direction) and the second set represents column iteration (y direction) 
and hence the name alternating direction. The term implicit in the name 
of the method is used to denote the fact that solutions at iteration n + 1 
cannot directly be obtained in terms of solutions at iteration n as was 
the case with overrelaxation methods. On the other hand when solutions 


u 
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at iteration n + 1 can be expressed in terms of solutions at n the method 
of solution belongs to the general category of explicit methods. 
It is suggested [1] to use the equation 


pict sek oh obese Sie a 


(bxF - }) 


y 1/(t- 1) 
xX = oat 
(;) 


for determining the values of the constant r. In the above equation a 
and b are the sides of the rectangle, k is the number of iterations, and t 
is the number of the last iteration. Since t is not known at the beginning 
of a given problem an assumption as to its value is made and the resulting 
r’s are computed. These values are then used in Eqs. (8.24). If the 
number of iterations t seems unsatisfactory from the convergence view- 
point of resulting solutions, another number is chosen and the calculations 
repeated. 

In general, it can be shown that the number of iterations needed to 
achieve specific solution accuracies is much less using the alternating- 
direction method than successive overrelaxation. However, this apparent 
advantage should be balanced against the difficulty of implementing 
calculations depicted by Eqs. (8.25) and the selection of parameter r. 


(8.27) 


SOLUTION OF THE HEAT EQUATION 


Consider the one-dimensional heat conduction equation 


a (8.28) 


where u(x,t) is the temperature distribution, x is the space dimension, 
and ¢ is the time. The constant a depends on the heat transfer properties 
of the solid in question. For simplicity, assume that a = 1. Using the 
relations 
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Ax? h2 
du af +1 1 
ot 7 


where / and T are the space and time increments Ax and At, respectively, 
Eq. (8.28) becomes 


Us gy = O45 + (1 — 2a) U; , + aU; 4; (8.29) 


Indices i and j are used for space and time dimensions, respectively. a in 
Eq. (8.29) is related to h and T by 


ere (8.30) 


Having the values of the temperature u at time jT, u(i, j) for all i’s, the 
values of the temperature at time (j + DT, u(i,j + 1), can be obtained 
using Eq. (8.29). This equation takes an especially simple form when 
a= 1/2, that is, 


1 
pe 5 1 * 41,7 (8.31) 


Or, the temperature at a point 7 at time j + 1 is equal to the average of 
the temperatures at either sides of 7, i+ 1,andi-—1, at timej. It can also 
be shown that the solution represented by Eq. (8.29) is stable only for 
values of a < 1/2 (see [2] and the following general discussion). The 
numerical solution of the heat equation (8.28), using the above method, 
is illustrated by the following example. 

Example. Consider the solution of the temperature distribution 
equation 


u(x,t) _ dulx, t) (8.32) 
x2 ot 
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with the following initial and boundary conditions: 


u(Q,t) = 100 constant temperature boundary at x = 0 


ale t) = 0 insulated boundary at x = 1 
ox 
u(x,0) = O initial temperature equal to zero 


In order to use matrix notation, the increments and indices are set up as 
in Fig. 8-5. The insulated boundary condition 


OPT he hg (8.33) 
Ox 


where T is the time increment, can be represented by using backward 
differences on the space coordinate x atx = nh 


oe Ng 
pe Ss REIL og (8.34) 
Ax 
or 
Sei en - 1g 


That is, when the temperatures of the nth and the (n — 1)th rows of 
Fig. 8-5 are equal, there will be no flow of heat between the regions 
= 1) he x 

Assuming that the region between x = 0 to x = | is divided into ten 
equal increments of h = 0.1, the value of n = 1/0.1 = 10. The initial and 
boundary conditions now become 


II 
cn) 
| 


me dig ey 
ie = 100 J = = a 3; seatie (8.35) 


Se 


Numerical Solution of Partial Differential Equations 323 


a 100 all j 
wegegs = 
Paar 0) ae 9, (42) Ct,:3) (1, 4) 

itt 2) 


x=ih 


all j 


u (ih,0) = 0 


c= (n—1) MUO) _flr= 1) fln=T, 2) 


ae ie (n, 3) 


figure 8-5. Solution of heat flow equation (8.32). 


du(nh,jT) = 0 alll j 


Having these values, Eq. (8.29) can be solved for u; oy for j = O and 
i=1,2,3,4,...,9. The value of v9 4 is set equal to Ug o by the last of 
Eqs. (8.35). Having these values, Eq. (8.29) can now be used to evaluate 
u’s at the next time step by setting j = 1 andi=1,2,3,...,9. The value 
of W109.1 = Yo 1 This procedure can be continued for j = 2, 3,...,andso 
on. 

Since the value of h was set at 0.1, a value of T= 0.005 will result in 


Using this value of a, solutions can be evaluated for t = 0.005, 0.01, 
0.015, and so on. These solutions were evaluated by the digital computer 
program given below, and the results are presented in Fig. 8-6. It is 
noted that as time increases, the solution approaches the steady-state 
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Termperature t 
wn 
ro 


NEN aE 
Ge Bee, SIRNS el ae Se 
tL 
densi oom [SL] | [ot 


=e = 0,4. 05 06-67 08 09-18 


Distance x 


figure 8-6. Solution of heat equation. 


value of 100. The digital computer program was written in BASIC 
programming language [3] to accomplish the above numerical procedure. 

Statement 10 reserves memory spaces for u. This statement assumes 
that there are ten divisions in the x direction and that the time extends 
for 160 increments. Statement 11 sets the value of the index M (time 
index) equal to 160. Statement 15 sets matrix U equal to zero. State- 
ment 20 sets the value of A = a = 0.5 for an h = 0.1 and T = 0.005. 
Statement 25 sets the value of index N = 1/h = 10. Statements 40 
through 85 implement the first two equations of (8.35). Statements 90 
through 120 carry out the numerical solution using Eq. (8.29) and the 


10 DIM U(10,160) 
11. LET M=160 
15 MAT U=ZER 
20 LET A=.5 

25 LET N=1/.1 
40 FORI=1TON 
50 ~=LET U(I,0)=0 
60 NEXT I 
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70 FOR J=O TOM 

80 LET U(0,J)=100 

85 NEXT J 

90 FOR J=-OTOM-1 

94 LET K=J+1 

95 FOR Il=1 TON-1 

100 LET U(I,K)=A*U(I+1,J)+U(1,J)*(1-2*A)+A*U(I-1,J) 
115 NEXT | 

116 LET U(N,K)=U(N-1,K) 
120 NEXT J 

132 FOR J=0 TOM STEP 20 
133. PRINT “J="J 

134 FOR I=OTOM 

135 PRINT U(I,J); 

136 NEXT | 

137 PRINT 

138 NEXT J 

140 END 


last of Eqs. (8.35). Statements 132 through 138 print the answers which 
are the elements of matrix U. These answers are printed for J = 0 (t = 0), 
J = 20 (¢ = 0.005 x 20 = 0.1), J = 40 (¢ = 0.2),...,J = 160 (¢t = 0.8) 
starting with 7 = 0, 1, 2,..., 10, as shown in Table 8-2. Note that the 
first value of U for each J is equal to 100, which is the boundary 
condition at x = 0. 


GENERAL FORM OF THE HEAT EQUATION 


The general form of the heat equation in one space dimension can be 
written as 


ve 
Ou _ 22 = 0 o = const > 0 | (8.36) 


ot ax? 


The boundary conditions u(x,t) and the initial condition u(x,0), are 
assumed to be available. This equation can be written in the following 
finite-difference form: 


po = et Oe Oe PO” 
—$ CS 


At (Ax) 


(8.37) 
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where i is the space and n is the time index (x = i Ax, t = n At) and the 
52 operator is used to designate 


2 ee k k 
ee = en ey 


(8.38) 


and @ is a positive constant between zero and one. Thus, Eq. (8.37) in 
the case of 6 = 0 specializes to 


~ sil =i. (*) Ces - 2u." + u?_,) = 0 (8.39) 


Using the notation 


At 


a= o6— 
Ax? 


Eq. (8.39) becomes 
ee oe, ¢ CL — Deda” + ont (8.40) 


This equation, but for indexing, represents Eq. (8.29). For any other 
value of 6 (6 #0) Eq. (8.37) will result in a set of simultaneous equations 
to be solved at each iteration. The condition for the stability of solution 
of Eq. (8.37) is, from [2], 


ee 0 <5 6 < 

2 = 
(An)? 1 - 20 (8.41) 
No restriction if > ee. epee 


It can be further shown that the error in representation (8.39) is of the 
order 


e(u) = O(At) + O[(Ax)?] (8.42) 
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By varying the values of @ in Eq. (8.37) a number of relations can be 
obtained for solutions u(x,t) (see [2]). One such solution is for 6 = 1/2 


oe as AS ee 


(8.43) 
At 2Ax? 


developed by Crank and Nicholson in 1947. It can be shown that this 
solution is always stable (O < 6 < 1) and the truncation error is of the 
order of magnitude 


e = O[(At)?] + O[(Ax)2] (8.44) 


Expansion of Eq. (8.43) will result in 


1 o At 
gH ae = aso — 2u; + u;_ 4)" (8.45) 


net 
te > pe a ) 


Noting that the values of u at time step n are available, the above 
equation can be put in the form 


Awe: = B,u;"** + Cet Stee ee Me 
(8.46) 


where M is the number of points in the x direction. And, since the 
boundary conditions are known, the equations representing the bound- 
aries will be of the form 


Peis 1 6%? re By 
(8.47) 


ntl n+l 
Ay Uy — 4 + Byuy = Dy 


The solution of Eqs. (8.46) and (8.47) can be obtained by the method 
of Eq. (8.21), at each iteration. Note that in formulations where 6 + 0 


Eq. (8.37) results in a set of simultaneous equations to be solved at 
each iteration. 
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Formulas similar to Eq. (8.37) can be derived for the solution of the 
two-dimensional heat equation 


2 2 
ee (8.48) 
ot Ox? oy? 
where A and C may be functions of x and y. Denoting 
uP = uli Ax, j Ay, n At) (8.49) 
and 
. A n y) n 92 n 
Wag acne eo Pia) Chis. - up + UE) 
(8.50) 
We can write the finite difference representation of Eq. (8.48) by 
yrti_ yn, 
BT oF See 14 = n 8.51 


where again 6 represents a number between zero and one. For 6 > 0 
the above equations result in a system of simultaneous equations to be 
solved at each iteration. For 6 = 0, of course, this is not required as the 
equations for u”*! will be in terms of uv” known quantities. The con- 
ditions of stability of solution of Eq. (8.51) are 


meee < Ped <2 
(Ax)2 (Ay)2—s 2 - «40 
No restriction if ; < §-s. 1 


This can be considered as generalization of stability condition found in 
the one-dimensional case. 
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SOLUTION OF THE WAVE EQUATION 


The wave equation, which represents the position of a vibrating string 
as a function of time (Fig. 8-7), can be represented as 


2 2 
ao eS (8.52) 


ax? ot? 


where y(x, é) is the vertical position at distance x and time ¢t and c? isa 
constant depending on the physical properties of the string. 


| 
i 


vil..t) <= OU 


x 


figure 8-7. Coordinates of wave equation. 


Assume the following initial conditions for the wave equation (8.52) 


y(x,0) = fx) 

(8.53) 
Ee Oe 6 
at 


These equations give displacement and velocity of the string at time 
zero. The assumed boundary conditions at x = O and x = L, where L is 
the length of the string, are as follows: 


y(0,f) = 0 
y(L,t) = 0 


(8.54) 


These equations state that the end points of the string are fixed. Let us 


denote the deflection at x; at time ¢; by y; ;, and the space and time 
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increments by h and T, respectively. Having done this, Eq. (8.52) can be 
written in central difference form (of order h”) as follows: 


1 x 
Ree i + 95 -1,5* Ee a4 og eae (8.55) 
a 
where 
a c* T2 
h2 


The value of y, j+ycan be obtained in terms of values of y at the previous 
time steps ei Eq. (8.55). This results in 


Ste. oa a*(y, eer ye atl a*)y, ee (8.56) 
For a = 1 (in [2] it is shown that solutions will be stable only if a < 1), 
this equation reduces to the simple form 


eas eee (8.57) 
Let us assume that the length of the string is divided into n parts 


where n = L/h. The initial and boundary conditions (8.53) and (8.54) 
can now be written 


Yio = f(x.) i = Oton 
Belg = y 
ee gy, be Vion 
2T . (8.58) 
Yo; = 0 ee ae 
Yn j = 0 fe 1 A 


In the second of the above equations, central differences are used to 
represent the initial zero velocity condition. 
Equation (8.56) for any i and j = O can be written 


2 | 
2 7 Was iae opp ta a”) ¥; 9 ane ee (8.59) 
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The second of Eqs. (8.58) states that y; _) = 9; 43 substituting in Eq. 
(8.59), we get 


2 
a 2 
Yi, {= es (y < 1,0 “f a 1.0) S (1 =a V9 (8.60) 


This equation can be used for obtaining the value of y id ffi = 1,2. 32. =, 
n—1. Note that the values of y, , and y, , are given by the boundary 
conditions (8.58). Having these values, and setting j = 1 in Eq. (8.56), 
the values of ¥; 5, = 1, 2, 3,..., 7—1 canbe obtained for ¢ = 2At = 2T. 
The procedure can be carried out for j =3,4,5,..., and so on. 

Example. As a numerical example of the above equations, consider 
the problem of the vibrating string with the initial conditions given in 
Fig. 8-7. Dividing the length of the string in n parts (h = L/n), the 
initial and boundary conditions (8.58) will become 


L 


tea = n(©) fet 2 i 2 
a 9 
= hip = (=) for <i <n 
n (8.61) 
oe 
Y0,j 0 
ee 


where b is the initial height at the center of the string. Assuming the 
length of the string L = 2, the initial height b = 1/2, and dividing the 
length into 10 equal parts (n = 10), we get 


2 
i re Th ee (8.62) 
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or 
j ae a 
Using Eq. (8.60) for i = 1 to n — 1 will give the values of Te for the time 


T = 0.2. Note that the value of y, , is zero by the boundary condition. 
After these values are calculated, Eq. (8.59) can be applied for j = 1 and 


i= 1,2,3,...,n-—1 to compute the positions of the string for j = 2 or 
time =t = 3 (0. 2)= 0.4. The solution can proceed by = Vp ; for 
j=3andi=1,2,3,...,n-—1andsoonforj=4,5,. 


The analytical Sclution of the vibrating string ein with the given 
initial and boundary conditions can be obtained from [5]. This solution 
is given in an infinite trigonometric series as follows 


(8.63) 


The numerical solution described above and the analytical solution 
(8.63) were evaluated with the aid of digital computers. The series of 
Eq. (8.63) was summed from m = 1 to 100. Table 8-3 shows the partial 
results of the calculations. An examination of this table shows that the 
values calculated by the numerical method are extremely close to the 
analytical solution. For a = 1, it can be shown that the numerical 
solution is identical to the analytical solution [6]. The slight discrepancy 
in the values as given in Table 8-3 is primarily due to the approximation 
in the evaluation on the series (8.63), that is, m to 100 instead of m = ~. 
The accuracy of the numerical solution decreases slightly as a assumes 
values smaller than one. 

For purposes of comparison, the series of Eq. (8.63) was also evaluated 
summing up to m = 5. The results are presented in Table 8-4 for time 
equal to 0.6. Note that the finite-difference solution of the table is the 
exact solution. 


SUMMARY 


The solutions given above were primarily derived for linear partial 
differential equations. However, the methods can be extended to non- 
linear partial differential equations. One such application is given in [7]. 
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Table 8-3. Results of the Solution of the Wave Equation 


DIST X | 

Y-ANLYT Y-FINITE DIFF 
0 0 0 0 
4 9.99999E-2 | .1 9.99998 E-2 4 
A 5 2 199999 2 
c 299999 3 299998 2 
8 399996 4 398986 4 
fe 497974 5 399996 4 
1.2 | 399996 4 398986 4 
1.4 | .299999 “ 299998 3 
1S Sle ee 5 199999 5 
1.8 | 9.99998E-2 | 1 9.99998 E-2 = 
i 14 91987 BS 78 ~ 1.30857 E-8 0 

Time = 1.0 Time = 2.0 
DIST X 

Y-FINITE DIFF 
0 0 0 0 
2 |-5.32606 E-10 |-1.04774E-9  |-9.99999 E-2 | 
4 1|-1.84689E-9 |-1.62981 E-9 Ee, 
6 |-3.14625E-9 |-2.09548 E-9 3 
8 |-3.82612E-9 |-2.79397E-9 -4 
{12567606 E98 * 1<2.32831 BO am 
1.2 |-6.65077E-9 |-2.79397 E-9 -4 
1.4 |-8.00339E-9 |-2.09548 E-9 a 
1.6 |-9.46699E-9 |-1.62981 E-9 ao 
1.8 |-1.08282E-8 |-1.04774 E-9 a4 
2-5 es Ela 1 1.31982 E-8 0 


Time = 0.2 


The stability and accuracy of the numerical solutions described above 
have been the subject of extensive study by mathematicians and engineers. 
Reference 1 contains several chapters that discuss the solution of partial 
differential equations. References 8 and 9 compare numerical and 
analytical solutions of specific partial differential equations. 


PROBLEMS 


1. Derive the expressions of Eq. (8.4). 
Divide Fig. 8-3 into four equal increments in each direction and 
formulate a table of coefficients corresponding to Table 8-1. Obtain 
the numerical solution and compare with the values given. 
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Table 8-4. Comparison of Solutions of the Wave Eq. (8.63) 
: Time = 0.6 


0 


DIST X 


Y-ANLYT 
0 


108262 
.180754 
.205959 
.201388 
195393 
.201388 
.205959 
.180754 
108262 
-1.47587 E-8 


.10099 

.189964 
.201149 
199561 
.200318 
199561 


.201149 
.189964 
.10099 
-1.17699 E-8 


Y-FINITE DIFF 
0 


RPNNNNNNNY 


© 


3. Solve Prob. 2 by Liebmann’s iteration method. 
4. Derive Eq. (8.17) for the case of Laplace’s equation (8.8) for # = 1/2. 
5. Derive Eqs. (8.20) from Eq. (8.19). 
6. Prove relations (8.22) for the set of equations 
B, T, - C,T, = D, 
fee ef, CT, = D, 
m7 7 BT. - dD. 
7. Solve the set of equations 
aI4 1 0 0 0 X, real 
1 =1.5 1 0 0 X, 0.1 
0 i eb Sck ol Oo bokac= 12.08 
0 0 i = 5 - X, 0.1 
0 () 0 1 -1.5] \X, oe | 


by the method of Eqs. (8.22) and (8.23). 
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8. Solve equations 
25 1 0 0} /X, a7 
a =46 3 O}] X, 45 
6 bi 41 oe 
0 0 1 5] \X, 14 
by the method of Eqs. (8.22) and (8.23). © 
9. Formulate the problem depicted in the example of Fig. 8-3 by the 
successive-overrelaxation method of Eq. (8.17) with » = 1.5. 

10. Formulate the problem depicted in Fig. 8-3 by alternating direction 
method of Eq. (8.24). Draw the logic block diagram of the solution 
for a digital computer program. 

11. Formulate the solution of the steady-state temperature distribution 
problem of the rectangle of Fig. P-11. 

u = 500 0 0 0 500 
1,000 1,000 
1 unit 1,000 1,000 
1,000 1,000 
500 0 0 0 500 
rt OO 

fig. P-11. 

The boundary conditions and space increments are specified. Use 
the alternating-direction method with r = 0.5. Obtain numerical 
values for one complete iteration. 

12. Obtain the vaiues for J = 5, t = 0.025 of Fig. 8-6 for the given heat 
transfer problem. 

13. Formulate the heat transfer problem depicted in Fig. 8-5, using the 
Crank-Nicholson formula (8.43). 

14. Formulate the heat transfer problem of Fig. 8-5, using four incre- 


ments in the X direction, by the Crank-Nicholson method of Eq. 
(8.43). Obtain numerical values for the first iteration. 


15. 


16. 
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For 6 = 1/2 formulate equations resulting from the expansion of 
Eq. (8.51). 

Obtain the finite-difference solution of the wave equation for t = 0.2 
as tabulated in Table 8-3. 
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9 LINEAR AND 
SEPARABLE 
PROGRAMMING 


This discussion is intended to give the reader a basic understanding of 
linear programming and enable him to recognize and formulate linear 
programming problems. The solution of these problems can be obtained 
by the simplex method, which is discussed briefly. For a more complete 
discussion, the reader is referred to [1]. The simplex method, depending 
on the size of the problem, involves a great number of arithmetic calcu- 
lations. Therefore, the method leads naturally to solution by digital 
computers. Actually the development of digital computers has greatly 
furthered the utilization of linear programming in the solution of many 
industrial problems. Today, practically in every industrial or research 
center having access to a digital computer one can utilize digital com- 
puter codes written for the solution of linear programming problems. 

The linear programming methods have been applied to the solution of 
technical as well as economic, scheduling, and allocation problems [1]. 
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The methods have been very successfully applied in the petroleum 
industry for the solution of gasoline blending problems [1] and economic 
optimization of oil well development and exploration [2]. The linear 
programming methods, however, are limited by the fact that, as the name 
implies, they can be applied only to the solution of problems whose 
fundamental characteristics are expressed by linear equations. This 
limitation has been somewhat reduced by the emergence of a number of 
methods that extend the region of applicability of linear programming to 
restricted nonlinear regions [3]. The simplest of these methods is the 
method of separable programming [4] which will be discussed in this 
chapter. This method is an extension of linear programming which allows 
the inclusion of nonlinear constraints in the formulation of the problem. 
The method of solution of these problems is identical to the simplex 
method of linear programming. 


FORMULATION AND METHOD OF SOLUTION 


Suppose that we have a set of n nonhomogeneous equations in n 
unknowns, as follows: 


Meee f- 1,2,...,7 (9.1) 


This set of equations has a unique solution for X ’s, provided that the 
determinant formed from the a, j coefficients is nonzero. Notice that in 
the above, the number of equations and the number of unknowns are the 
same, namely, n. Now, consider the case where there are more equations 


than unknowns, i.e., 
Ftp =i a (9.2) 


with n > m. In this case, as long as the equations of the set are inde- 
pendent, there is no solution at all to the system of equations. Finally, 
consider the case where there are more unknowns than equations: 
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Se fas 10, (1 2S (9.3) 


with m >on. In this case there is no unique solution to the system of 
Eqs. (9.3). Introducing the additional condition 


» a,,, = minimum i = 0 (9.4) 


and the restriction that the solutions should result in x. > O (negative 
values not accepted) may lead to a unique solution of the linear program- 
ming type or, with these restrictions, there may not be a solution to the 
system at all. In the latter case the linear programming problem is 
termed infeasible. In summary, the set of relations 


bs Q;;X; = b. = 1S... (9.5a) 
j=1 
» a,;x, = minimum i = 0 (9.5b) 
j=l 

iD jf sei fsa (9.5c) 


with m > n, constitute a linear programming problem. Equations (9.5a) 
are called the constrains or the restraint equations while Eq. (9.5b) is the 
functional and is also referred to as the objective form. Equation (9.5c) 
is the nonnegativity requirement on the variables. Observe that the 
number of nonzero x.’s given by the linear programming solution will be 
equal to the number of constraints. 

To become familar with the terminology and the simplex method 
of solution of linear programming problems, consider the following 
numerical example: 
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3x, - 2x, < 2 (9.6a) 
+ 2x, < 4 (9.6b) 
X, + 2x, 2 6 (9.6c) 
-~3x, - Xx, = F = minimum (9.6d) 
- = =, > 0 (9.6e) 


Equations (9.6a), (9.6b), and (9.6c) are the constraints, while Eq. 
(9.6d) represents the functional to be minimized. It will be seen later 
that the inequality constraints (9.6a)-(9.6c) are converted to equality 
constraints by addition of slack variables. Addition of these variables 
will increase the number of unknowns and render the problem into the 
form where the number of the unknowns is greater than the number of 
equations. (If the functional is to be maximized, the coefficients of the 
variables in the functional are multiplied by —1. Then the minimization 
of the resulting functional is equivalent to maximizing the original 
functional.) Equation (9.6e) gives the nonnegativity requirement on the 
variables x, and x,. Note that all of the constraints have positive 
right-hand sides. The above constraints and the functional are represented 
graphically in Fig. 9-1. In this figure, the constraint 3x, - 2x, < 12 
represents the region to the left of the line AA, the constraint 
—x, + 2x, < 4 represents the region below the line BB, and the constraint 
x, + 2x, 2 6 represents the region above the line CC. Thus, the region 
covered by small squares represents the allowable region for the solution 
of the problem. The dashed lines are for various values of the functional 
F. From this figure, the constrained minimum of the function F occurs 
at x, = 8 and x, = 6 and has a value of F = —30. Now let us solve the 
same problem by the simplex method. 

First, replace the inequality constraints (9.6a), (9.6b), and (9.6c) by 
equality constraints. This is accomplished by adding slack variables 
Xg, X4, X5, and xg to the system, as follows: 


3x, — 2X9 + Xg 2092 (9.7a) 


—X1, + 2x5 + 2X4 = 4 (9.7b) 
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figure 9-1. Simplex method for the example problem. 
X41; Xo, X93, X45 X55 X6 = 0 (9.7) 


Note that Eqs. (9.6a) and (9.6b) are “‘smaller than” inequalities and they 
can be changed to equalities (9.7a) and (9.7b) by adding positive slack 
variables x, and x,. On the other hand, Eq. (9.6c) is a “greater than” 
inequality and it can be changed to an equality by adding a slack 
variable x, with a —1 coefficient. But this, as will be seen later, is not 
computationally desirable. Therefore, add another slack variable Xn 
with a +1 coefficient. At this point the constraint (9.6c), represented by 
equality (9.7c), will be satisfied only when x; = 0 and x, 2 0. Thus, as 
long as x, 2 0 we will not be in the allowable region of Fig. 9-1 and the 
solutions will be somewhat artificial.* Equation (9.7d), as before, repre- 
sents the functional to be minimized, and Eq. (9.7e) is the nonnegativity 
requirement on all of the variables. 


*Slack variables performing functions similar to that of X. are sometimes called 
artificial slacks. In some problems, in order to derive these variables out of the final 
solution, they are included in the functional with arbitrarily large coefficients 
(penalty coefficients). 
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Start the simplex method by taking the solution 


which makes F = O for the first iteration. Observe that this solution 
satisfies the constraints ((9.7a), (9.7b), and (9.7c)) and the nonnegativity 
requirement (9.7e). In linear programming, solutions of this type, having 
the same number of nonzero variables as there are constraints, are called 
basic solutions, and the nonzero variables are said to be in basis. At this 
point the question is posed—can the functional assume a lower value? 
The answer is yes, since, in the functional (9.7d), x, and x, have negative 
coefficients. Therefore, increasing the value of either one from zero 
(given by iteration 1) will decrease the value of the functional. In 
particular, since x, has a larger negative coefficient, a higher rate of 
decrease of the functional can be obtained by increasing x,. The amount 
that x, should increase can be obtained from the following observations. 
Increasing the value of x, in a particular constraint will result in an 
increase, a decrease, or no change in the values of the other variables 
present in the constraint. We need not concern ourselves with variables 
which increase or remain unchanged, but should be careful that de- 
creasing variables do not become negative. This might arise only in 
constraints in which x, appears with a positive coefficient. Thus, we 
only examine constraints (9.7a) and (9.7c) of the problem. From Eq. 
(9.7a), bringing x, in basis will decrease the value of x, (which is in 
basis in iteration 1). The highest value that x, can have (with x,= Q) is 
12/3 = 4. Similarly, from Eq. (9.7c) x, can attain a value of 6 with 
x= = 0. Thus, take the smaller of these values, 4, and bring x, in basis 
through the constraint (9.7a). Dividing Eq. (9.7a) by the coefficient of 
x,, we obtain the set of equations 


x, - 2x5 + 3x3 = 4 (9.8a) 
-X1 + 2X5 + X4 = 4 (9.8b) 
X1 + 2X» + Xp — Xe = 6 (9.8c) 
-3XxX,- Xo = F (9.8d) 


a, Xo, Nag Rave Megs to. (9.8e) 
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Eliminating x, from Eqs. (9.8b), (9.8c), and (9.8d) by multiplying Eq. 
(9.8a) by proper constants and adding to Eqs. (9.8b), (9.8c), and (9.8d) 
results in 


X1- 2X» + +x3 = 4 (9.9a) 
=i + aXg + X4 =o (9.9b) 
=e = aX + Be + Kes = (9.9c) 

-8X5 + Xg = 398 (9.9d) 


The basic solution of this iteration is 
x, = 4 Xo = V X_g = O x, = 8 Xp = 2 Xe = O 
This results in 


F+12 = 0 
or 


ps 212 


Thus, in this iteration the value of the functional F is decreased by 12 
units. Further decrease of the functional can be obtained by increasing 
X5, which has a negative coefficient in Eq. (9.9d). Consider constraints 
(9.9b) and (9.9c), in which x, appears with a positive coefficient. From 
Eq. (9.9b) we note that x, can attain a limiting value of (8)/(4/3) = 6 
and from Eq. (9.9c) a limiting value of (2)/(8/3) = 3/4. Thus, we bring 
%5 in basis through constraint (9.9c). Dividing Eq. (9.9c) by 8/3, we have 


1 - Bxq + 4X5 = 4 (9.10a) 
4x9 + 4X3 +°X4 = 8 (9.10b) 
Xo — iXg + Sx5 - 3X6 = 3 (9.10c) 


=O + Xe = F412 (9.10d) 
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Eliminating x» from Eqs. (9.10a), (9.10b), and (9.10d), we get 


Xy + 4X3 + 4X5 = iX¢ = 
5X3 + X4 - 9X5 + 3% = 

oe Sy, - x, 

2x3 + a Xs - 2 x6 = 


The basic solution of the above iteration is 


This solution gives 


i —2t aot 35 


9 

2 

7 

3 

4 
po OL 
SS 
ca 


(9.1 1a) 
(9.11b) 


(9.11c) 
(9.11d) 


Observe that in this iteration x, iS Zero and the solution is no longer 
artificial. We refer to this solution as a basic feasible solution. From 
Eg. (9.11d) we observe that the functional F can be further decreased by 
increasing xg, which appears with a negative coefficient. Equation (9.11b) 
gives the largest value that x, can have, that is, (7)/(1/2) = 14. Thus we 
bring x, in basis through Eq. (9.11b) by eliminating xg in Eqs. (9.1 1a), 


(9.11c), and (9.11d). This results: 


1 1 a 


Xe + 2X4 -%5 + Xe = 


1 3 
X59 a ——s + 3 %4 = 


7 9 


8 


14 


6 


F’ + 20 


The basic feasible solution of this iteration is 


(9.12a) 


(9.12b) 


(9.12c) 
(9.12d) 


14 
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This solution results in a functional F = —30. Since negative coefficients 
do not appear in Eq. (9.12d), the basic feasible solution 


x, = 8 X_ = 6 and ie = 


with a functional of F = —30 is the optimal solution. 

The iterations together with the corresponding functional values are 
expressed in Fig. 9-1. The first iteration with x, = 0 and x, = 0, F = Ois 
at point (1); the second iteration with x, =4, x. =0, F =—12 is at point 
(2); and so on. Observe that each iteration moves the solution from one 
corner to the other, while decreasing the value of the functional. Note 
that, since at iteration 3 the solution becomes feasible, point (3) is a point 
of the allowable region. 

The simplex method, as described above, and its variations are used for 
the solution of linear programming problems. The simplex method can 
easily be coded for digital computers. The rules to be followed in the 
solution of linear programming problems by the simplex method can be 
summarized as follows: 

1. Find the x, with the highest negative coefficient in the functional. 

2. Locate the constraints in which x, has positive coefficients. 

3. Divide the right-hand side of these constraints by the respective 

x. coefficients. 

4. Choose the smallest of the above values and the respective con- 

straints. 

5. Divide this constraint by the coefficient of x, and eliminate x, 

from all of the other constraints and the functional. 

6. Repeat the above procedure. 

The discussion of the linear programming method has thus far covered 
only the basic principles of the method. The theory and the computa- 
tional aspects of linear programming have developed considerably beyond 
the presentation of this chapter. The interested reader is referred to 
[5, 6] for further study of the subject. 


SEPARABLE PROGRAMMING 


Separable programming is a generalization of the simplex method of 
linear programming for certain types of nonlinear problems. This 
generalization of the method was developed by C. E. Miller and presented 
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In the above equations n,’s are parameters. The first two equations 
((9.13a) and (9.13b)) give the values of x and y asa function of param- 
eters n,. Equation (9.13c) together with the requirement that no more 
than two of the n.s can be nonzero and these must be consecutive will 
restrict the x and y values to the line segments. The meaning of this will 
become clear when we express Eqs. (9.13) numerically for y = f(x) of 
Fig. 9-2. Reading the values of x, and y, from the figure, we have: 


x = n,(1) + n(2) + ng(4) + n4(6) + ng (9) + ng (10) (9.14a) 
y = n,(1) + ny(4) + n3(6) + n4(7) + ng (6) + ng (5) (9.14b) 
T= fy + Mg + Ny + By + Hy + He (9.14c) 
Ny, No, Ng, Ng, Ng, Ng = 0 (9.14d) 


For example, a solution satisfying Eqs. (9.14c) and (9.14d) and the 
consecutive n, requirement will be nz = 0.5, and 4, = 0.5 wie 
N, = Ny = Ng = ng =O. This gives 


x = 0514) + 0D = 9 
(913) 
y = 0.5(6) + 0.5(7) = 6.5 : 


This is, as expected, a point on the line segment v,v,. Another solu- 
tion, for example, satisfying Eqs. (9.14c), (9.14d), and the consecutive 
n, requirement will be n, = 1.0 with the rest of n;’s equal to zero. This 
corresponds to the point (4, 6) of Fig. 9-2, which is the end point of a 
line segment. We emphasize again that in the linear programming formu- 
lation n,’s represent a set of parameters that are treated as unknowns. 
Note that the condition specified by Eq. (9.14d) need not be incorporated 
in the formulation, since the method of linear programming deals only 
with positive or zero values of the unknowns. 

The above method can be extended to three or more dimensions. 
Three-dimensional relationships, geometrically expressed by surfaces, can 
be replaced by a number of triangular planes approximating the surface. 
For example, the surface f(x,y, z) = 0 can be approximated by a number 
of triangular planes, and equations corresponding to Eqs. (9.13) can be 
written: 
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x= ) n,x, (9.16a) 
i=1 

: ej = 9; (9.16b) 
i=1 

225) (9.16c) 
i=1 

i > », (9.16d) 
i=l 

n, > 0 (9.16¢) 


In the above equations, x,, y,, and z, represent the coordinates of the 
vertices of triangular planes, andn,’s are parameters. Together with the 
condition that no more than three (as compared with two of the previous 
case) of the n,’s can be nonzero and these must be associated with a 
triangular plane approximating the surface, Eq. (9.16d) will restrict the 
x, y, and z values given by Eqs. (9.16a), (9.16b), and (9.16c) to the 
triangular plane. This becomes clear by taking the example of a surface 
approximated by only one triangular plane with vertices (1, 1, 1), 
(2, 2, 3), and (1, 2, 2), writing the equation of the plane passing through 
these, and showing that the points obtained from Eqs. (9.16) are located 
on this plane. 

The general equation of a plane is written 


Ax + By + Cz = 1 (9.17) 


Substituting the x, y, and z values at the vertices, we obtain a set of three 
equations and three unknowns, as follows: 


II 
—" 


feos se 
(9.18) 


| 
— 


9A +96 + 3C = 
oo OU 2 Lk 
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Solving the above equations for A, B, and C(A = 1,B=1,C = -1D 
and substituting in Eq. (9.17), we obtain the equation of the plane 
passing through the vertices 


Ly =— 2g = 1 eed, 


On the other hand, Eqs. (9.16) can be written for the triangular plane 
with the specified vertices as follows: 


X = n,( + No (2) a ng (1) 


y = n, (1) = Ny (2) + Ng (2) 


= Dw BABY a) (9.20) 


1 = Nn, + No + Ne 
N4, Nog, Ng 2 0 
For example, the values of n, = 0.4, n, = 0.3, and ng = 0.3 will 
satisfy the last two equations as well as being then,’s associated with the 
triangular surface. The corresponding values of x, y, and z obtained from 


Eqs. (9.20) should be on the plane described by Eq. (9.19). Substituting 
the specified n,’s in Eqs. (9.20), we have 


Caer es 
y = 1.6 
eee 


which, indeed, satisfy Eq. (9.19) and hence are on the triangular plane. 
The above principles have been used extensively in the formulation of 
the following problem. 


FORMULATION OF A NUMERICAL PROBLEM 


The formulation of separable programming problems is illustrated by 
two numerical examples. The problems selected are to find the optimum 
operating conditions in a power plant with three generators each with 
different power-fuel consumption characteristics [7]. 
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Case 1 


The first case is that of a power plant with three boiler-turbine- 
generator combinations with nonlinear generated power vs. fuel consump- 
tion characteristics. Two of the boilers use fuel oil and the third uses 
coal. The power plant load, combined outputs of the three generators, 
and the fuel costs are specified. The objective is to find the operating 
level of each generator to minimize total fuel costs. 

The generated power-fuel consumption characteristics are given in 
Fig. 9-3. The A.’s, p;’s, and v.’s specify points on the curves of the 
boiler-turbine-generator combinations A, p, and v, respectively. The 
nonlinear curves are replaced by piecewise linear functions shown in the 
figure. For example, the curve of the boiler-turbine-generator combi- 
nation v is replaced by linear segments vj,v5, vgvg, Vgv4, and vygvs. 
Using these segments, the following equations for the generated power, 
fuel consumption, and the sum ofn,’s can be written: 


25n, + 23n, + 19.5n, + 16.7n, + 13n, = G., (9.21a) 
$.40n, + 7.86n, + 6.99n, + 6.45n, + 5.87n, = F, (9.21b) 
N, + No + Ng +n, +n = 1.0 (9:21c) 


where G, and F,, are the generated power and coal consumption of the 
boiler-turbine-generator combination v. Note that the last equation, 
(9.21c), along with the additional requirement that only two consecutive 
n;s are to be nonzero will guarantee a solution on one of the specified 
line segments, as discussed previously. In the cases examined, the 
consecutive requirement was automatically satisfied, mainly because the 
generated power-fuel consumption curves were concave upward,* as 
noted by the following two observations. First, consider generator v 
of Fig. 9-3 operating at 18 MW; the corresponding fuel consumption on 
line v3v, is lower than on any other line passing through two v’s, for 
example, vov5. Second, consider the main objective of the formulation 


*This argument does not apply where the generated power-fuel consumption 
curves are concave downward—a condition rarely encountered in power plant opera- 
tions. In these cases the additional provisions (specified in [4] ) dealing with consecu- 
tive nonzero n, and m, values should be incorporated in the linear programming code. 
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Boiler—Generator A and p 
use fuel oil 


Boiler—Generator V 


Fuel consumption, tons/hr ° 


Generated power, MW 


figure 9-3. Generated power vs fuel consumption for Case 1. 


which is to minimize the amount of fuel (within the specified constraints). 
From these it is easy to see that at any operating level, the solution will 
seek the lowest obtainable fuel consumption, thereby satisfying the 
consecutive n, requirement. 

Similar equations can be written for the other two boiler-turbine- 
generator combinations A and p. The parameters associated with these 
are 1 and m, respectively. Note also that these use fuel oil as the boiler 
fuel. 

After completing these calculations, the linear programming matrix of 
the problem can be constructed as given in Fig. 9-4. The matrix contains 
a listing of the unknowns, the constraint equations, and the functional, 
the latter representing the objective function to be minimized. The right- 
hand side (RHS) of the equations is given on the extreme right of the 
matrix. The linear programming matrix of Fig. 9-4 was solved by a 
computer code using the simplex method of solution. 

In the matrix, G’s and F’’s refer to generated electrical power and fuel 
consumption, respectively. The /’s, m’s, and n’s specify parameters 
associated with the linearized curves of Fig. 9-3. Constraints 1 to 3 are 
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for boiler-turbine-generator combination A, 4 to 6 for np, and 6 to 9 for v. 
The constraint equation of row 10, 


G.+G,+G, = 54.0 MW (9.22) 


specifies the total load on the power plant. The functional represents the 
cost of fuel consumed by the plant. It is assumed that fuel oil costs 
12.6 $/ton and that coal cost 10 $/ton. Thus, the functional becomes 


12.6F, + 12.6F + 10F,, = minimum (9.23) 


as given on the last row of the matrix. 

The solution of the linear programming matrix gave the numerical 
answers shown on Fig. 9-4. The corresponding operating levels are 
specified by ((-)) on Fig. 9-3. Note that generators A, », and v operate 
at 9, 21.9, and 23 MW, respectively. The total fuel cost (functional) 
is 169.5 $/hr. 


Case 2 


The second case is that of a power plant with two boiler-turbine- 
generator combinations, each capable of using a mixture of two types of 
fuel oil. The fuel consumption-load characteristics are nonlinear, while 
the fuel consumption-type of fuel characteristics are a linear function of 
the fuel blending ratios. The total electrical output of the power plant 
and the amount and type of fuel available at zero cost are specified. The 
Object is to minimize the amount of purchased fuel. 

The generated power-fuel consumption characteristics of this case are 
given in Fig. 9-5. Note that they can be considered as surface with fuel 
constituting the third dimension. The boiler-turbine-generator combi- 
nations ’ and p» represent systems the boilers of which can use type-A or 
type-B fuel oil or any combination of these. For example, \ operating at 
20 MW will use 5.5 tons per hour of type-B fuel oil or 5.65 tons per hour 
of a 1:1 ratio of type-A and type-B fuel oil. Connecting three consecutive 
r’s or p’s, we will obtain triangular planes for each of the two sets of 
surfaces. Thus, in the present case, we can have three nonzero values of 
/. Or m; aS Opposed to two nonzero values in the previous cases. We 
should be careful, however, that the nonzero J, and m, values in the final 
solution represent a plane which is the closest approximation to the 
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figure 9-5. Generated power vs fuel consumption for Case 2. 


generated power-fuel consumption surface passing through the same /,’s 
or m;’s. From the results of the cases examined, because of the concave 
upward nature of the surfaces this condition was automatically satisfied. 

After obtaining the coefficients from Fig. 9-5 we can set up the 
linear programming matrix of Fig. 9-6. The last two constraints specify 
the amount of available fuel at zero cost (10 tons of type-B fuel) and the 
desired power plant output (50.0 MW). The objective is to minimize the 
amount of purchased type-A fuel oil. F,, and F, . Tepresent type-A fuel 
used in generator-turbine-boiler combinations » and _ p, respectively. 
Similarly, F,, and F'p . represent type-B fuel used in generator-turbine- 
boiler combinations 4 and pz. 

The solution gave the results reported in Fig. 9-6 and the corresponding 
operating points specified ((-)) in Fig. 9-5. Note that generators A and pu | 
operate at approximately 23.5 and 26.5 MW, both below their rated 
capacities. The total type-A fuel oil consumption was 3.17 tons/hr. 
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PROBLEMS 


1. Obtain the solution of the following linear programming problem by 
the simplex method: 


eS 
4 +=1, <= 6 


x4 + 4x. — maximum 


Show that the solution occurs at the intersection of the lines 


2x, - Xp = 8 
Answer: x, = 6, ta 4. 


2. Obtain the solution of the linear programming problem 


oc, < 0 


—X, + 2x5 <4 
Answer: %.-= 4,233; 45> 4.117. 
sty 2 4 


X, + Xp = maximum 


3. Write the equation of a plane passing through (1, 1, 1), (3, 4, 6), and 
(2, 3, 1) and equations corresponding to Eq. (9.20) of the text. 
Show that the point n, = 0.2, n, = 0.2, and nz = 0.6 is on the plane. 

4. Formulate the separable programming matrix for the case of a 
power plant with two generator-turbine-boiler combinations A and 
uw with the following characteristics 


Fuel consumption, Generated power, 
tons/hour megawatts 
6 10 
Generator A 7 15 


10 20 
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Fuel consumption, Generated power, 
tons/hour megawatts 
5 10 
Generator pL 6 15 
10 20 


It is required to generate 35 megawatts, minimizing fuel consump- 
tion. 
5. Obtain the solution of the linear programming problem 


II 
II 


%1 — Xo + 2X9 =e 2 Answers: x1 = Os Xo 


Go = 45 

= 3 oe 
2X, + Xo — 3xg + x, = 6 
X1 + X» + Xy +X, = 7 


~2X4 -X > + X3 +X, = Maximum 


6. Obtain the solution of linear programming problem 


2x, + Xs a Answers: i, = 3 
x, + 3X5 <= sy 
< 
2X 5 <> 4 
3X4 — X 9 = maximum 


7. Write the logic for a digital computer program for the simplex 
method. 
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70 DYNAMIC 


PROGRAMMING 


Dynamic programming has been successfully applied to optimizing 
returns from a set of engineering units, operating in series or in parallel, 
to produce certain outputs. In chemical engineering applications, the 
chemical process of production can be divided into a series of stages 
(units) where the output of one stage constitutes the input of the 
next stage. In industrial engineering applications, the productian process 
can be considered to be accomplished in a series of units each requiring 
a certain amount of effort for a specified return. The problem of finding 
the most economical operating levels of electrical generators (each with 
different efficiency-load characteristics) in a power plant can be formu- 
lated by dynamic programming. 

This discussion is intended to give the reader an understanding of the 
basic principles of dynamic programming. Numerical examples of parallel 
and series operation of production units are given. The results obtained 
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for the parallel case are compared with the solutions obtained by the 
method of separable programming described in the previous chapter. 


GENERAL DYNAMIC PROGRAMMING ANALYSIS 


Consider a certain engineering production operation consisting of n 
units, as shown in Fig. 10-1. For convenience in calculation, number the 


Unit 21%2 *1 
d9 
(x) 


(return) 


ES ee ee 


Total return R 


figure 10-1. Dynamic programming configuration of production units. 


last unit (on the right) as unit | and the first unit on the left as unit n, 
and assume the following notation: 


= input to unit 7 


y; = output of unit 
; = decision required at unit 7 
R. = return of unit i 


In the above notation, x,, y,, and d; can represent vectors. This means 
that input and output may consist of several components and there can 
be several types of decisions possible at each unit. Using the principle of 
dynamic programming, the total return R can be optimized (maximized 
or minimized) as follows. 

Starting with unit 1, optimize the return of this unit R, with respect 
to its input x,, output y,, and decision d,. Select the optimum operating 
points of unit 1 and combine this with operating points of unit 2. 
Optimize the combined returns of units 1 and 2. Select the corresponding 
optimum operating points. Combine these points with operating points 
of unit 3 and calculate the corresponding returns. Select the optimum 
operating points of unit 3 and the corresponding returns. Continue this 
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process. This procedure is governed by the “‘principle of optimality” of 
dynamic programming [1]: 


No matter what the input (x) and decision (d,) and resulting 
output (y) at unit 7 may be, the succeeding decision, d; , v 
must be made in such a way as to yield the optimum return 
from the 7 + 1 units. 


Let us formulate the mathematical expression of the above principle for 
the serial multistage process of Fig. 10-1. Let us further assume that the 
input to each stage is equal to the output of the previous stage, i.e., 


eS (10.1) 
For general stage i of the n-stage system of Fig. 10-1, we can write the 


expressions for the output and the return as a function of the input to 
stage i and the decision made at stage 7. 


Xe 4 t.(x.,d,) 


(10.2) 


en 
L 


r(x .,d-) 


II 


where x,_, is the output of stage i (the input of stage — 1) and ¢, and r; 
represent functions of variables in parentheses. Since the output of every 
stage could be expressed as a function of its input and the stage decision, 
starting from stage n and proceeding to n— 1,n-2,... and finally to 
i + 1 which supplies the input of stage i, we can write 

of Gas4) (10.3) 


so ES 
which represents the input of stage i as a function of system input x, and 
the decisions made up to stage i. Using this expression in the return 


(Eq. (10.2)), we can express 


oa ® (10.4) 


ee 


In comparing Eqs. (10.3) and (10.4), we see that the decision of stage 
i,d,, affects only the return of this stage and, of course, later stages. 
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The total n-stage system return R will be a function of individual 
returns from each stage; using Eq. (10.4), the total return can be 
expressed 


R = 3 Ee Se fo x, d., a ee et ee 


wow Ka - TO 


ae 
(10.5) 


eS: 


The optimization problem will require the maximization or the minimi- 
zation of the above return as a function of system input x, and stage 
decisions d,,d,_,,---,d,. Using Eq. (10.2) for stage returns, the total 
return can be expressed also as 

(x d 


= or (ed), r ee ee (10.6) 


n-l‘“n-1? “n-1°’ 


Consider maximization of return R where 


3 ee ee d+ -+- +7, be, d,) (10.7) 


n-1 
represents the sum of individual stage returns. Using the optimality 
principle and denoting by R, the optimum i-stage return (stages | through 
i), we write 


ee i d) + R,_4] (10.8) 
The above is the recursion equation of dynamic programming. Starting 
from i = 1 and proceeding to i = 2, 3,...n the total return of i stages are 
maximized as a function of stage 7 decision d;. Thus, the total n-stage 
system maximization problem is reduced to maximizing returns from 
1, 2, 3,... , stages, respectively, and combining the cumulative return 
of (i — 1) stages with the ith stage, and so on. As will be seen in the 
following examples, use of the methods of dynamic programming greatly 
reduces the number of calculations required in a particular optimization 
over the number which would be required if one had to choose the 
optimum from all of possible combinations of decisions. 


NUMERICAL EXAMPLES 


The following numerical examples will help to clarify the above 
principle. 
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Example of Parallel Operation 


Consider a power plant with three boiler-turbine-generator combina- 
tions A, B, and C, with the cost vs. load curves given in Fig. 10-2. From 
this figure the values of cost for various values of generated power, for 


Operating cost, dollars /hr 


©-Operating pts. for 60 MW 


Generated power, MW 


figure 10-2. Operating cost vs. load for generators A, B, and C. 
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each of the three generators, can be obtained and tabulated. This 
tabulation for increments of 5 MW is shown in Table 10-1. Suppose that 
we want to generate a total of 60 MW in the most economical way and 
that we proceed to formulate the problem using dynamic programming. 


Table 10-1. Generated Power and Cost for Generators A, B, and C 


Generator A Generator C 


Generated Cost, $/hr Generated Cost, $/hr Generated Cost, 
power, MW power, MW 
30 10 


power, MW 
10 40 


10 

15 40 15 46 15 

20 51 20 35 20 a4 
66 > 65 i 


90 30 78 30 


Let us take generator A operating at 10 MW and B operating at 10, 
15, 20, 25, and 30 MW, respectively, and tabulate the values given in 
set 1 of Table 10-2. The first row of this set gives the cost of generating 
20 MW, 10 by A and 10 by B, as 70 $/hr, which is the sum of cost 
figures from Table 10-1 (30 + 40 = 70 $/hr). Ina similar manner, A 
operating at 10 MW and B at 15 MW will cost, from Table 10-1, 30 + 46 
= 76 $/hr, as given in the second row of the first set in Table 10-2. 
Hence, we can complete the rest of the values of Table 10-2 with the aid 
of Table 10-1. Table 10-2 has, in fact, all of the possible operating 
combinations of generators A and B. 

Up to this point we have merely tabulated values. At this time we 
utilize the governing principle of dynamic programming to reduce the 
number of computations when adding the output and costs of generator 
C to the respective values of generators A plus B as given in Table 10-2. 
An examination of Table 10-2 shows that there are several combinations 
of generators A and B that will result in the same amount of total power. 
For example, we can generate 35 MW by operating A at 10 and B at 25, 
as given in the fourth row of set 1; or A at 15 and B at 20, as given in the 
third row of set 2; or A at 20 and B at 15, as given in the second row of 
set 3: or A at 25 and B at 10, as given in the first row of set 4. But 
from all of the mentioned combinations for generating 35 MW, the 
combination of A at 10 and B at 25 with a total cost of 95 $/hr, as 
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Table 10-2. Operating Combinations of Generators A and B 


Set A operating B operating Cost(A + B) | Total power 
at MW at MW $/hr (A + B),MW 


10 10 8 
10 15 a 
1 10 20 10h 
10 25 oa” 
10 30 40 
15 10 Zo 
15 15 30 
2 15 20 35 
15 25 40* 
15 30 45 
20 10 30 
20 15 3D 
3 20 20 40 
20 22 45* 
20 30 50* 
25 10 ee 
25 15 40 
4 2 20 45 
Pde fda: 50 
25 30 oe 
30 10 40 
30 15 45 
5 30 20 50 
30 a 55 
30 30 60* 


given in the fourth row of set 1, is the most economical.* In a similar 
manner we can pick up the most economical ways of generating 20, 
25, ..., 60 MW from Table 10-2 as marked by asterisks. From now on 
we will be concerned only with the most economical combinations and 
costs of operating generators A and B and we can, in effect, drop all of 
the other values. Thus, in incorporating the generated power of generator 
C we use the combinations noted by asterisks in Table 10-2. These values 
are listed in Table 10-3. In this table we have two additional columns—one 


*Note that 35 MW can also be generated at 95 $/hr by operating A at 15 andB 
at 20 as given in the third row of set 2. In case of ties either combination can be 
used for further study. 
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Table 10-3. Operating Combinations of Generators A + B and C 


Generated power 
(A + B),MW 


for the load on generator C and the other for total cost. Since the total 
generated power as given in the problem is to be 60 MW, we calculate 
the load of generator C on this basis. For example, when A + B are 
generating 20 MW, C should be generating 60 — 20 = 40 MW. But this, 
from Fig. 10-1, is outside the operating range of C, so that no numbers 
are entered in the first row of the third and fourth columns. When 
A + Bare generating 40 MW, C will be generating 60 — 40 = 20 MW;; the 
cost will be 105 + 51 = 156 $/hr, where 51 $/hr is the cost of C operating 
at 20 MW. | 

The rest of the values of Table 10-3 can be computed in the same 
manner. From the total cost column of this table it is seen that the 
152 $/hr gives the most economical way of operating the generators for 
generating 60 MW of power. We see that the most economical operation 
corresponds to C operating at 30 MW and A + Bat 30 MW. Returning 
to Table 10-2, we obtain the operating levels of generators A and B which 
give the total of 30 MW. This value is given on the third row of set 1, 
and corresponds to A operating at 10 MW and B operating at 20 MW. 
These points (A at 10, B at 20, and C at 30) are marked on Fig. 10-1. 

Now let us ask ourselves what we have gained by using dynamic 
programming. First, if we were to do this problem by considering all 
possible operating combinations of generators A, B, and C and correspond- 
ing costs, we would have 5 x 5 x 5 = 125 combinations to consider (each 
generator can operate at 5 levels, that is, 10, 15, 20, 25, and 30). With 
dynamic programming we have reduced the number of calculations to 
25 (in Table 10-2) + 5 (in Table 10-3) = 30. This is a reduction of over 
75 percent. Second, we have followed a logical pattern for performing 
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computations that can easily be programmed for a digital computer. The 
saving in computation will become more evident if we consider a finer 
grid for operating ranges of generators. For example, if we consider an 
increment of 2 instead of 5 MW we will have 11 load values for each 
generator, that is, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, and 30 MW. The 
total number of possible operating combinations will be 11 x 11 x 11= 
1,331. By dynamic programming the number of combinations will be 
reduced to 11 x 11 + 11 = 132 —a reduction of over 90 percent. 

The solution given by Table 10-3 not only gives the most economical 
operating levels, but also gives other operating levels with increased cost. 
Sometimes it is desirable to operate at a level that is more practical than 
economical. For example, if we do not want to operate any of the 
generators at their rated capacity (30 MW), we might consider the solution 
corresponding to the total cost of 154 $/hr as given in Table 10-3. This 
solution results in operating A at 10, B at 25, and C at 25 MW; but of 
course this will cost us 154 — 152 = 2 $/hr more than the optimum 
solution. 

Now we ask ourselves whether if we had considered generators C and 
B first and then added generator A we would still get the same answers. 
The answer is that we will get the same optimum answer (A at 10, 
B at 20, C at 30 with a cost of 152 $/hr), but the other combinations 
will not be the same. This can be verified by taking generators C and B 
and completing a table similar to Table 10-2 and then incorporating 
generator A and completing a table similar to Table 10-3. This was done, 
and the final table of computations is shown in Table 10-4. A comparison 
of Tables 10-3 and 10-4 shows that optimum operating combinations 


Table 10-4. Operating Combinations of Generators C + B and A 


Generated power Cost (C + B) A, MW Total 
i +B), MW $/hr eo 5 &) cost 


20 (C at 10, B at 10) 
25 (C at 15, B at 10) 


30 (C at 20, B at 10) 30 181 
35 (C at 20, B at 15) 25 163 
40 (C at 25, B at 15) 20 156 
45 (C at 30, B at 15) 15 153 
50 (C at 30, B at 20) 10 ee ke 


55 (C at 30, B at 25) 
60 (C at 30, B at 30) 
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and corresponding costs are the same in the two tables, but the inter- 
mediate operating combinations and costs are not the same. For example, 
from Table 10-4, the next lowest cost to the optimum is 153 $/hr while 
in Table 10-3 the corresponding value is 154 $/hr. 

We solved the load allocation problem and presented the results in 
Table 10-3 for the total power generation of 60 MW. Using the first two 
columns of Table 10-3 we can find the most economical generating levels 
of the three generators for any other value of total generated power. The 
results of these calculations are illustrated in Fig. 10-3. From this 
figure, for example, we observe that the most economical ways of 
generating 75 MW and 80 MW are operating A at 20, B at 25, and Cat 
30 at a cost of 182.5 $/hr and operating A at 20, B at 30, and C at 30 at 
a cost of 195 $/hr, respectively. This figure can also be used for obtain- 
ing operating levels that are not multiples of 5. For example, if we want 
to generate 78 MW at minimum cost, this value can be obtained from 
Fig. 10-3 by operating A at 20, B at 30, and increasing the operating 
level of C to 25 + 3 = 28 MW. The cost, by linear extrapolation, will 


250 


Minimum cost, dollars /hr 


Total generated power, MW 


figure 10-3. Most economical power generation levels of generators A, B, and C 
for required total power generation. 
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be approximately 182.5 + 3/5 (192 — 182.5) = 190 $/hr. Of course, a 
more accurate answer can be obtained by solving the problem with 
load increments of 1 MW. 

To compare the above solutions with the solutions obtained by other 
methods, we will solve the problem by the method of separable program- 
ming. 


Separable Programming Formulation 


The problem described above can also be solved by separable program- 
ming, as discussed in the previous chapter and in [2]. In formulating the 
problem for this method we need to substitute piecewise linear approxi- 
mations for the nonlinear cost-load curves of Fig. 10-2. This approxi- 
mation is shown by dashed lines in the figure. The equations for 
generated power and cost can be written in terms of “special variables”’ 
which incorporate the piecewise linear approximations of the nonlinear 
curves. The equations for generator A can be written as follows: 


=P + Wi, + 175A, + 26.25 


II 
oS 


-C, + 30A, + 45A, + T0A, + BOA, (10.9) 


A, tAgtrAg + Aq = 1 


where A,, Ay, Ag, and A, are the special variables and their coefficients in 
the first two equations are the coordinates of the intersections of line 
segments describing the curve of generator A in Fig. 10-2. The first two 
equations give the amount and cost of generated power, respectively. 
The last equation, with the additional condition that only two of the ’s 
can be nonzero and that these must be consecutive, will guarantee a 
solution on one of the specified line segments. Equations similar to 
Eq. (10.9) can be written for generators B (with special variables ») and 
C (with special variables y). The separable programming matrix of the 
problem is shown in Fig. 10-4. Rows 1 through 9 of the matrix give 
constraint equations for generators A, B, and C, while row 10 specifies 
the total required amount of generated power, i.e., 


P, + Pp =< Po = 60 (10.10) 


In Fig. 10-4 the designation RHS specifies right-hand side of equations. 
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The solution of the problem by the simplex method of linear program- 
ming is shown at the bottom of the figure. This solution results: 


Generated power, MW Cost, $/hr 
P,=10 30 
Re 55 
P= 30 67 
Total 60 152 


Note that the above solution is exactly the same as obtained by dynamic 
programming. 

The method of dynamic programming as applied to the above problem 
is simple to follow and the calculations involved are logical and can be 
performed by digital computers with a small amount of programming 
effort. The accuracy of the solution can be improved, if necessary, by 
taking smaller increments for the generated power. This will increase the 
number of calculations to be performed although the logical sequence of 
calculations will remains the same. 

The solution obtained by separable programming will depend on the 
accuracy of representation of cost-load curves by piecewise linear seg- 
ments. We note that this accuracy can be easily increased by taking more 
linear segments to represent a given curve. This in turn will increase 
the number of special variables, but fortunately an increase in the number 
of variables does not appreciably increase the computation time. Hence, 
by separable programming we have a greater control on the accuracy of 
the solution. In addition, solutions can be obtained for any value of 
required total generated power whereas this will require finer increments 
of generated power in dynamic programming. The separable program- 
ming method, of course, will require a computer code (usually the 
simplex method) for solution, while dynamic programming does not 
involve any special method. 


Example of Series Operation 


Consider the three-stage production process of Fig. 10-5. The products 
manufactured at stage 1, x), have a market value of $10 each. These 
products can be manufactured from input products x,, using one of three 
processes. The input-output relationships of these processes are given in 
Fig. 10-6 as a function of types of process (decision) d,,d5, and dg. 
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figure 10-6. IJnput-output relationship of stage 1 with decision, d, 
as parameter. 


These decisions have the following cost schedule: 


Cost of processing each unit of x, with decision d, = $10 

Cost of processing each unit of x, with decision d, = $5 

Cost of processing each unit of x, with decision d= $5 : 
Cost of implementing decision d, = $500 

Cost of implementing decision d, = $700 

Cost of implementing decision d, = $600 


The input-output relationship of stage 2 is given as a function of the 
type of process (decision) in Fig. 10-7. In stage 2 two types of decisions, 
with the following cost schedule, are possible: 
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Output xy 


figure 10-7. Input-output relationship of stage 2 with decision, d, 
as parameter. 


Cost of processing each unit of x, with decision d,= $10 
Cost of processing each unit of x, with decision d, = $5 
Cost of implementing decision d, = $600 

Cost of implementing decision d, = $500 


In stage 3 two types of decision are possible. The cost schedule of 
these decisions is as follows: 


Cost of processing each unit of x, with decision d, = $10 
Cost of processing each unit of x, with decision d, = $20 
Cost of implementing decision d, = $500 
Cost of implementing decision d, = $100 


With the above data it is desired to obtain the optimum decisions of each 
stage for producing 100, 200, 300, ..., 1,000 units of x) for maximum 
return. 
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For the solution of the problem consider stage 1 and obtain the values 
of input x, for each value of x, and decision d,. These values are given 
in Table 10-5. Asan example of calculating the values of this table, con- 
sider output x, of 500 units and decision dy. From Fig. 10-6, an output 


Table 10-5. Input and Return of Stage 1 


Input x, /return, $ Input x, /return, $ Input x, /return, $ 


100 19/ 310 5/275 10/ 350* 
200 28/1,220 10/1,250 20/1,300* 
300 | 35/2,150 16/2,220 30/2,250* 
400 | 40/3,100 213,195 40/3,200* 
500 | 43/4,070 27/4,165* 50/4,150 
600 | 45/5,050 34/5 ,130* 60/5,100 
700 ) 47,6,030 41/6,095* 70/6,050 
800 | 48/7,020 50/7,050* 80/7,000 
900 49/8,010* 62/7,990 90/7,950 
1,000 50/9,000* 80/8 ,900 100/8,900 


of 500 units corresponds to an input x, of 27 units using decision d, 
process. Since the selling price of each unit of x, is $10, the total return 
of stage 1 will be 


R = 500 x 10 - (27 x 5 + 700) = $4,165 


where 5 is the cost of processing each unit x, with decision d., and 700 
is the initial cost of implementing decision d,. The stage return and the 
number of inputs required are separated by a slash (/) in Table 10-5. 
After completing the values required in this table we choose, for each 
output x,, the most profitable operating decision-highest return. These 
values are marked with an asterisk. From now on we need only concern 
ourselves with these values. 

Moving to a two-stage process consisting of stages 1 and 2, we tabulate 
the values of x, and the values of x, corresponding to the most profitable 
operation of stage 1. Columns 1, 2, and 3 of Table 10-6 represent these, 
values along with the corresponding return of stage 1. Columns 4 and 5 
give the corresponding number of input units x, obtained for the output 
units x, of stage 2 obtained from Fig. 10-7. These columns also include 
the combined two-stage return. As an example of calculations, consider 
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the calculation involved for x) of 800 and decision d, as represented in 
column 5 of Table 10-6. For an x, of 800, the corresponding x, from 
Table 10-5 is 50 with a return of $7,050. From Fig. 10-7, an x, = 50 
corresponds to an input to stage 2, x,, of 50 using decision d,. The 
corresponding return of the two-stage process is calculated as follows: 


R = 7,050 — (50 x 5 + 500) = $6,300 


where 5 is the cost of processing each unit of x, and 500 is the cost of 
implementing decision d,. These costs are subtracted from the one-stage 
return of $7,050. After completing Table 10-6 the returns are optimized 
with respect to decisions and distinguished by an asterisk. 

The return of the three-stage process can be computed in a similar 
manner and a table of values obtained. As was the case with the values 
of Table 10-6, only the asterisked values of Table 10-6 will be used in 
the compilation of the three-stage process (Table 10-7). This table gives 
the most profitable two-stage values in columns | through 4 as obtained 
from Table 10-6. The three-stage returns are calculated for decisions d, 
and d, of the third stage. For example, for a value of x) = 700 
decision d, will result in the three-stage return: 


R = 5,390 — (41 x 10 + 500) = $4,480 


where 41 is the number of stage-three outputs required, 10 is the cost of 
producing each unit, and 500 is the initial cost of implementing 


Table 10-7. Three-stage Return 
x4 X5 Two-stage 
units | units return, $ 

10; 10 -200 
700 
1,600 


Decision d 1 


Three-stage return, $ 


Decision d, 
Three-stage return, $ 
-500* 


200* 
900* 
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decision d,. The most profitable three-stage returns are marked by aster- 
isks. Thus Tables 10-5 through 10-7 give the most profitable decisions 
for producing quantities of x). The results can be combined and written 
in a single table, as in Table 10-8. This table gives the required decision 
at each stage and the resulting profit for producing various quantities of 


Xo: 


From this table it is seen that manufacturing 100 units of x9 will 


incur a loss of $500, while manufacturing 1,000 units will bring a profit 
of $7,250. 


PROBLEMS 


1. 


Find the optimum level of operating generators A, B, and C of the 
example problem for generating 60 MW by taking combinations of 
generators A and C first, and combining the results with generator 
B. 

Find the optimum level of operating generators A, B, and C of the 
example problem for generating 80 MW of total power. Check your 


answer with the value given in Fig. 10-3. 
3. Repeat the above problem for 70 MW of total generated power. 


4. In the three-stage production problem, assume the following cost 


figures for the first stage: 


Table 10-8. The Overall Decision Process and Return of the Three-stage 
Series Example 


Output Overall 
X Stage 1 Stage 2 Stage 3 Profit, 
es 1 eee ae ar df 8 
100 / / J -500 
200 V / s/ 200 
300 / / s/ 900 
400 J J / 1,600 
500 / / / 2,890 
600 J J / 3,680 
700 / s/ / 4,480 
800 / J / 5,300 
900 | JV / J 6,275 
1000 | V ees 7,250 
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Cost of processing each unit of x, with decision d, = $8 
Cost of processing each unit of x, with decision d, = $4 
Cost of processing each unit of x, with decision d, = $2 
Cost of implementing decision d, = $500 
Cost of implementing decision d, = $500 
Cost of implementing decision da = $400 


and obtain the optimum production decisions as was done in the 
example problem. 

Taking the cost figures and curves of the three-stage production 
problem, calculate the most economical decisions for producing 
Xs in 50, 150, 250, 350, and 450 units. 

In the three-stage production problem, drop decision d, from con- 
sideration in stage |. With only decisions d, and d, remaining, 
calculate three-stage return for producing 100, 200, 300, and 400 
units of xp. 
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APPENDIX 
DIFFERENTIAL EQUATIONS 


DEFINITIONS 


The term ordinary differential equation is given to an equation contain- 
ing one or more ordinary derivatives or differentials; if the derivatives are 
partial derivatives, the equation is called a partial differential equation. 
For example, 


ete 2 ee (A.1) 


is an ordinary differential equation, while the equation 


ou Pes eV 


+ 4 = 0 (A.2) 
Ox? dy” dz 
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is a partial differential equation. Note that in Eq. (A.1) the variable x is 
a function of ¢ only, while in Eq. (A.2) the variable V is a function of 
x, y, and z. 

The order of a differential equation is the order of the highest derivative 
involved in the equation. For example, Eqs. (A.1) and (A.2) are both of 
the second order. 

The general solution of an nth-order differential equation contains n 
arbitrary constants. Consider the solution of the first-order differential 
equation 


dx 


dt 


= 4-4 2x (A.3) 


Separation of variables results in 


= (A.4) 


Integration gives 


In(l + 2x) = t+C (A.5) 


i 
2 


in which C is the arbitrary constant. The constant C can be evaluated 
using the initial condition. For x = 1/2 at t = 0, we get 


In2 (A.6) 


ee : (A.7) 
The general solution of the second-order differential equation 

2 

Peat 25 (A.8) 
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is given by 
x = Asinwt + B cosat (A.9) 


where A and B are the two arbitrary constants. The fact that Eq. (A.9) 
is a solution of Eq. (A.8) can be verified by obtaining the second 
derivative of x with respect to time and substituting in the differential 
equation (A.8). Given the initial conditions, the constants A and B can 
be evaluated. 

A linear differential equation is one in which the dependent variable 
and its derivatives appear only in the first power. The differential 
equations 


2, 
ee ee (A.10) 
dx? dx 
and 
Z 
ey + pa + (cosx)y = 0 (A.11) 
dx? dx 


are linear, while the differential equations 


2 
ay + 3(2) + Zy = 0 (A.12) 
dx? dx 
and 
2; 
eae = 0 (A.13) 
dx? dx 


are nonlinear. 

The principle of superposition of solutions applies in the case of 
linear differential equations. This principle can be stated as follows. If 
y = f(x) and y = g(x) are solutions of a linear differential equation, then 
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y = f(x) + g(x) is also a solution. It can be seen by substitution that 
y = Ae-*, where A is constant, is a solution of Eq. (A.10). Again by 


~ 


substitution, it can be seen that y = Be~**, where B is a constant, is also 
a solution of Eq. (A.10). By the superposition principle y = Ae~* + Be7?* 
contains two constants A and B and will be the complete solution of the 
second-order linear differential equation (A.10). 


LINEAR EQUATIONS OF THE FIRST ORDER 


Consider the first-order differential equation 


id _ Py = Q (A.14) 


where P and Q are functions of independent variable x. We wish to 
obtain the solution y in terms of x. Equation (A.14) can be written as 


dy + Pydx = Qdx (A.15) 
Multiplying Eq. (A.15) by R, an unknown function of x, we get 


Rdz + yRPdx = RQdx (A.16) 


Comparing this equation with the identity 


Rdy + ydR = d(Ry) (A.17) 
we get 

dR = RPdx (A. 18) 
and 

d(Ry) = RQdx (A.19) 


From Eq. (A.18) we obtain 


= .. Pac, B= en( fra) (A.20) 
R 
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From Eq. (A.19) we obtain 
Ry = J reas +c (A.21) 


where C is an arbitrary constant. Since Eq. (A.20) gives the value of R 
as a function of x, Eq. (A.21) will give the solution y as a function of x 
and the arbitrary constant C. 

As an example of the above, consider the solution of the differential 
equation 


es (A.22) 
Comparing this to Eq. (A.14) we have 
2 3 
Poo -= , Q = %& (A.23) 
xX 


which results in 


Rk = oo( f-2 ix) = exp (Inx~) 2 9? (A.24) 
x 


Using this in Eq. (A.21) will give 


x ey = fe dx 6 (A325) 
= care i 
Or 
Joa <= OY (A.26) 


where C is the arbitrary constant. 
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LINEAR EQUATIONS WITH 
CONSTANT COEFFICIENTS 


Consider the linear differential equation 
aD" y + “see +-+-+a@,_,Dy+ay =R (A.27) 


where a is a constant and D is the differential operator, D = d/dx, and R 
is a function of the independent variable x. Equation (A.27) can be 
written 


Bisy = Kh 
(A.28) 


fer ae? a pe apt os ea 


SOLUTION FORR = 0 


Consider the case of Eq. (A.27) when R =O and n= 1. This will result 
in 


fre )y = 0 (A.29) 
wherer, = a,/a,_, = 4,/d). Equation (A.29) can be written 


dy 


Sip 3 (A.30) 
dx 


resulting in the solution 


ee = C. exp (rx) , (A.31) 
Generalizing this, if the roots of the equation f(D) = 0 are distinct and 
equal to le fon ss 5 T,, then the solution y becomes 

y = ae Bie « ee - Cee oe = | (A.32) 


As an example of this consider the second-order differential equation 


Ae, 2% (A.33) 
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The roots of 


Ws = 2p 2 PS 1 


(A.34) 
= (2D — 1)(D + 1) 
arer, = 1/2,r, = -1. This results in the solution 
eee ae (A.35) 


If the roots of the equation f(D) = 0 are multiple roots with multiplicity 
n such that the differential equation becomes 


0 ee, eee (A.36) 


then the solution can be shown to be 
y = (Cy + Cox + Ce — BE es (A.37) 


As an example of this consider the differential equation 
2 
oe 5, (A.38) 
resulting in an f(D) of 


f(D) = D* ~6D +9 


. (A.39) 
= (D - 3) 


with multiple roots r, =r, = 3. The solution of Eq. (A.38) according 
to Eq. (A.37) will be 


y= (C, + Ce (A.40) 
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If the roots of the equation f(D) = O are complex conjugate roots 
m, = a+ jb andm, = a — jb, then the solution 


2 Coe (A.41) 


can also be written as 


~ 


— ie sin bx + K,e* cos bx (A.42) 


where C,, C,, K, and Kg are arbitrary constants. 
For example, the differential equation 


= 
a, Gy 0 (A.43) 
dx? dx 


with the roots of D? + 6D + 25 = 0 being equal to-3 + 4j and-3 - 4j 


will have a solution y = coc eos4dx + her sin4x. 


Solution for R 7 0 
For R # Othe differential equation (A.27) can be written in the form 
f(D)y = R (A.44) 


The solution of the differential equation (A.44) in this case will be the 
sum of two solutions y, and yz where y, is the solution of 


f(D)y, = 0 (A.45) 
and yo is the solution of 

f(D) y, = R (A.46) 
In Eq. (A.45) the value of R is set equal to zero. The solution of Eq. 
(A.45), y,, is called the complementary function while yo, the solution 


of Eq. (A.46), is called a particular solution. The solution of Eq. (A.45) can 
be obtained by methods described previously. The solution of Eq. (A.46) 
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can be obtained by assuming y, to be proportional to the sum of func- 
tions appearing in R (for example, x”, e?*, singx, cosux) and their 
derivatives as will be described in the following examples. 
Example 1. Solve the differential equation 

(D* + 2D-4 1y = 4e” (A.47) 
The equation corresponding to Eq. (A.45) can be written 

a 2) 2 Dy, = 0 (A.48) 
This results in the complementary solution 


a= 3 Ce (A.49) 


The particular solution y, of Eq. (A.46) is obtained by assuming an 
exponential function of the type given in Eq. (A.47),R = 4e%, 


Yoo = Ae (A.50) 


Obtaining appropriate derivatives and substituting in Eq. (A.47), we 
obtain 


Ae~ + 2Ae* + Ae* = 4e* (A.51) 
which results in A = 1. Thus the complete solution of Eq. (A.47) becomes 
P= Yo = (C, + Coxde* +.€* (A.52) 
where C, and Cy are the two arbitrary constants of the second-order 


differential equation (A.47). 
Example 2. Solve the second-order differential equation 


24 
ee 4 a, ae (A.53) 
dx* dx 


The complementary solution is obtained from the equation 


(D2 - D)y, = 0 (A.54) 
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as 
y, = Cy + Cye* | (A.55) 


Considering the right side of Eq. (A.53) the particular solution y, will 
‘contain functions of the same form, 


x“, x, constant, e”, sinx (A.56) 


We note that the constant term also appears in solution y, of Eq. (A.55) 
as C,. For this reason the powers of x appearing in Eq. (A.56) should 
be multiplied by x, 


Yo = Wee Bx + Cx +. (A.57) 


where A, B, and C are constants. The term e* of Eq. (A.56) also appears 
in the complementary solution y, of Eq. (A.55). This term should also 
be multipled by x resulting in 


eee ee ee 2 Cx + Exe* + --. (A.58) 


Adding the term sinx and its derivative cosx, we can write the particular 
solution as 


5 = Ax” + Bx? + Cx + Exe* + F sinx + G cosx (A.59) 


Obtaining the derivatives of the above equation and substituting in Eq. 
(A.53), we will obtain an equation containing the constants A, B,C, ... 
and the functions appearing in Eq. (A.59). Equating the coefficients of 
the like terms, we can solve the resulting simultaneous equations for the 
values of the constants, 

(A.60) 


oe Cs CP oe 


1 
3 
The complete solution of differential equation (A.53) now becomes 


oe + Coe” = x° — x7 —~ 7x + 2xe* — s sinx + = Cosx (A.61) 


where C, and C, are constants. 
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SYSTEM OF LINEAR EQUATIONS 


The sets of simultaneous differential equations can also be solved using 
the methods discussed above. Consider the set of differential equations 


° = = dye 
x x 
(A.62) 
a 
dx 


where y and z are functions of the independent variable x. Equation 
(A.62) can be written as 


(D - 4)y + Dz = 1 (A.63) 


Gy a3) 9-4 2. 2x 


Multiplying the second equation by D and subtracting from the first we 
get 


(= 4D Dy = © 8 y = oe 5 (A.64) 
The complementary and particular solutions of Eq. (A.64) are as follows: 
¥, = (Cy + Cax)e** (A.65) 


and 


(A.66) 


(A.67) 


Solving the second of Eqs. (A.63) for z and substituting the value of y 
and its derivative from Eq. (A.67), we get 


z= (C, -C, + Coe = Sx +3 (A.68) 
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As a second example, consider the solution of the sets of equations 


. (2 = «) = 0 
dt? dt 
: (A.69) 
= (o . ) =A 
dt? dt 
The above equations can be written 
D?x + 2Dy = 2a 
(A.70) 
2x — D*y = 2b 
Solving the above for x, we obtain 
(D? + 4D)x = 4b (A.71) 
The solution of Eq. (A.71) is 
x = C, + Cy sin2t + C3 cos2t + dt (A.72) 


Using the second derivative of Eq. (A.72) in the first of Eqs. (A.69) 
we get 


Dy = 2C, sin2t + 2C, cos2t + a (A.73) 
Integrating this we obtain 
y = -C, cos2t + Cy sin2dt + at + Cy (A.74) 


Equations (A.72) and (A.74) with the four constants C,, Co, C3, and C, 
constitute the solution of the two second-order differential equations 
(A.69). 
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PROBLEMS 


Fr. 


Show that y = A sinwx + B coswx is the solution of the differential 
equation 


Show that y = A sin(wx + B) and y = C cos(wx + D) are the solu- 
tions of the differential equation 


Solve the differential equation 


ee ee | 


dx 


and check your solution by substituting in the differential equation. 
Solve the differential equation 


(D* —~ 6D2 + ly = 0 


Obtain the solution of differential equation 


Ss 
oS 4 


dx? dx 


= 


Answer: y = C, +C, cos2x + Cy sin2x 
Obtain the solution of the differential equation 
(Ps Ws Wie te 


Answer: y = C,e728* + Coe74* + 1Le2x 
Z I 2 15 


10. 


11. 


iz. 


Differential Equations 


Solve the differential equation 

(D? + D-— 12)y = xe* 

Answer: y = ee + ae = ee SE eee 
Solve the differential equation 


(D? —~ 2D + l)y = 2xe* 


Hint: The particular solution contains the terms xe* and x2e*. 


Solve the differential equation 


(D* + 4D3)y = x2 


x3 x4 x 


ee se C C.x + 0x" + —— = + 
= ———- + 792 198 om 


Solve the differential equation 
(D2 + 4)y = sin2x 
1 


Answer: y = Ci sin2x + Cy cos2x — z% cos 2x 


Solve the set of simultaneous differential equations 


dy 
— -6x+2y = 0 
dt? 
d2 
5 Br = 0 
dt? 
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by Milne’s method, 226 
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First-order differential equations, 237, 383 
FORTRAN program: 
for determinant evaluation, 8 
for matrix inversion, 32 
Forward difference operator, 186 
Forward differences in boundary value prob- 
lems, 284 
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Interpolation, inverse, 124, 145 
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Newton’s, 208 
Sterling’s, 209 
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Inverse matrix, 20 
eigenvalues of, 68 
eigenvectors of, 68 
by partitioning, 34 
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Krylov’s method for characteristic poly- 
nomial, 57 

Kutta, Runge, and Heun, integration formu- 
las of, 245 

Kutta-Runge method, 252 

Kutta’s fourth-order formula, 248 

Kutta’s method, 245 
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Lagrange’s interpolation formula, 117, 142 
Laplace’s equation: 
Gauss-Seidel method for, 316 
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implicit methods for, 317 
iterative methods, 311 
Liebmann’s method, 311 
relaxation factor in, 316 
solution of, 308 
Laplace’s operator, 309 
Laplace’s partial differential equation, 306 
Least-squares methods, 117, 145 
computer program for, 150 
Liebmann’s solution of Laplace’s equation, 
311 
Lin-Bairstow’s method, 134 
computer program for, 138 
Linear algebraic equations, 1 
Linear differential equations, 382 
Linear partial differential equations, 306 
Linear programming, 338 
basic solution of, 343 
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Matrices: 
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adjoint, 20 
algebra, 12 
calculation of Fourier transform by, 162 
characteristic, 51, 54 
column, 13 
conformable, 16 
continued products of, 17 
definitions, 13 
diagonal, 107 
division of, 20 
eigenvalues of, 51 
equality of, 14 
FORTRAN program for, 32 
inverse (see Inverse matrix) 
inversion of, 27 
iteration of, computer program for, 77 
iteration method for characteristic vectors, 
73 
modal, 106 
multiplication of, 15, 18 
properties of, 17 
notation, 13 
orthogonal, eigenvalues of, 66 
partitioned, 18 
postmultiplication, 17 
premultiplication, 17 
row, 13 
skew-symmetric, eigenvalues of, 66 
spectral, 106 
square, 13 
symmetric (see Symmetric matrices) 
transposed, 13 
eigenvalues and eigenvectors of, 103 
multiplication of, 18 
reversal law of, 18 
unit, 15 
zero, 14 
Mean values, 264 
Method of least squares (see Least-squares 
method) 
Milne’s method, 226, 239 
predictor-corrector formula, 227 
Minors of determinants, 2 
Mixed boundary conditions, 280 
Modal matrix, 106 
Monte Carlo methods, 264 
Multiple eigenvalues, 69 


Multiplication of matrices, 15, 18 
properties of, 17 


Navigation problem, proportional, 254 
Newton’s formula for coefficient of char- 
acteristic polynomials, 55 
Newton’s interpolation formula, 208 
Newton’s method for polynomial equations, 
125 
Nonlinear differential equations, 382 
Nonlinearities, simulation of, 260 
Nonoscillatory solution, 299 
Noumerov’s method, 241 
Numerical solution of differential equations, 
214 
errors in, 294 
stability of, 231, 290 


Objective form (functional), 340 
Operators: 
difference (see Difference operators) 
difference-differential, 187 
Optimality principle, 362 
Order of differential equation, 381 
Ordinary differential equation, 214, 380 
Orthogonal matrix, eigenvalues of, 66 
Orthogonality relations in obtaining charac- 
teristic vectors, 83 / 
Orthogonalization process, Gram-Schmidt, 
104 
Overrelaxation methods, 316 


Parabolic partial differential equations, 306 
Parallel operation in dynamic programming, 
364 
Partial derivative, 380 
Partial difference operators, 307 
Partial differential equation (see Differential 
equations, partial) 
Particular solution: 
of difference-differential equation, 293 
of differential equation, 387 
Partitioned matrices, 18 
inverse matrix by, 34 
Poisson’s equation, 315 
Polynomial equation, Newton’s method for, 
125 
Polynomials: 
characteristic (see Characteristic polyno- 
mial) 
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Polynomials (Cont'd) 

derivatives of by synthetic division, 121 

integration formulas for, 205 

quadratic factors of, 134 

roots of (see Roots, of polynomials) 
Postmultiplication of matrices, 17 
Predictor formula, 227 
Premultiplication of matrices, 17 
Programming, dynamic (see Dynamic pro- 

ramming) 

separable, 346, 338, 370 
Properties: 

of determinants, 4, 21 

of matrix multiplication, 17 
Proportional navigation problem, 254 


Quadratic factors of polynomials, 134 
Quadratic forms, 49 
Quasi-linear partial differential equation, 306 


Random inputs, simulation of, 260 
Random numbers, generation of, 262 
Recursion equation of dynamic programming, 
363 
Reduction of nth-order differential equation, 
218 
Relaxation factor in Laplace’s equation, 316 
Repeated eigenvalues, 69 
Restraint equations, 340 
Reversal law of transposed matrices, 18 
Roots: 
of equations, 117 
of polynomials, 118 
computer program for, 138 
Graffe’s method, 127 
Horner’s method, 124 
Lin-Bairstow’s method, 134 
Newton’s method, 125 
Row matrix, 13 
Runge, Heun, and Kutta, method of, 245 
Runge-Kutta method, 252 


Second-order differential equations: 
Milne’s method, 239 
Noumerov’s method, 241 
Separable programming, 346, 338, 370 
Series operation in dynamic programming, 
Siz 
Similarity transformation, 67 
Simpson’s integration formulas, 207 
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Simpson’s one-third rule, 203, 207 
Simpson’s three-eighths rule, 248 
Simulation: 
of nonlinearities, 260 
of randon inputs, 260 
Simultaneous boundary value differential 
equations, 286 
Simultaneous differential equations, 237, 
252, 390° 
Simultaneous equations, 1 
in boundary value problems, 272, 276 
Gauss’ method for, 21 
Gauss-Seidel method for, 37 
by matrix inversion, 26 
triangulation method for, 23 
tridiagonal, 318 
Skew-symmetric matrix, eigenvalues of, 66 
Slack variables, 341 
Smallest characteristic value, 81 
Spectral matrix, 106 
Square matrix, 13 
Stability of numerical solutions, 231, 290 
Standard deviation, 264 
Starting solution of differential equations, 
215 
Sterling’s interpolation formula, 209 
Successive displacement method, 316 
Successive-overrelaxation methods, 316 
Superposition principle, 382 
Symmetric matrices, 64 
diagonal form of, 108 
eigenvalues of, 65 


Synthetic division, 120 
derivative of polynomial by, 121 


Tabulated functions: 
derivatives of, 200 
integral of, 201 
Taylor series, 188 
Taylor’s method, 215, 218 
Thomas’ method, 318 
Time-frequency analysis, 157 
Transcendental equations, 125 
Transposed matrices, 13 
eigenvalues and eigenvectors of, 103 
multiplication of, 18 
Trapezoidal rule of integration, 202, 206 
Tridiagonal simultaneous equations, 318 
Two-dimensional heat equation, 329 
Two-point boundary value problem, 270 


Unit matrix, 15 


Vectors: 
characteristic (see Characteristic vectors) 
Vibration problems, 98 


Wave equation, 306, 330 


Zero matrix, 14 


